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Foreword 


Cedefop has been at the forefront of developing robust skills anticipation 
methods and skills intelligence tools for the European Union for more 
than a decade. The European skills forecast and the European skills and 
jobs survey shed light on how the labour market, skill needs and jobs are 
developing and help signal potential skills bottlenecks. Cedefop’s big data 
analysis of online job advertisements provides detailed and real-time skills 
intelligence capturing which skills have currency in job markets. Cedefop 
has used skills foresight to develop stakeholder-backed policy roadmaps 
aimed at strengthening national skills anticipation and matching systems. 
Complementing quantitative skills analysis and intelligence, qualitative 
insight on skills policies and measures also contributes to evidence-based 
policy-making. 

The continuing development of national skills intelligence systems 
and approaches has helped strengthen the feedback loops between the 
labour market and vocational education and training (VET) and skills policy. 
In the coming years, we need to be more ambitious. Our vision for Skills 
intelligence 2.0 is information that is more actionable: detailed and relevant, 
better contextualised, timelier, and better communicated. Making sense of 
trends and fostering capacity to act on them means combining sources and 
approaches - skill surveys, skills forecasting, skill foresight, big data analyses, 
and others — and exploring synergies. This gives policy-makers the means to 
separate noise from signal and supports employers and citizens in making 
decisions in line with the new realities in the world of work. 

It is no surprise that skills intelligence is a key priority in the 2020 European 
skills agenda. Reliable and fit-for-purpose labour market and skills intelligence 
has enormous value in times of rapid change and transformation. In a context 
of fast-paced digital advancements, such as artificial intelligence and 
advanced robotics, and other megatrends, such as population ageing and 
the green transition, VET and skills policies should become more proactive. 
To prepare new generations of learners and to support people in making and 
shaping career transitions, reliable skills intelligence is indispensable. 
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Foreword 


This publication is the third in a series of practical skills anticipation 
guides for policy-makers and analysts. The guides present a rich mosaic of 
conventional and emerging methods for identifying technological change and 
its impact on skills. Systematically presenting the merits and disadvantages of 
different methods, they show no single approach can provide all the answers. 
Apart from reliable data and sound methods, creativity, holistic thinking and 
using collective wisdom to shape the future are key building blocks of skills 
intelligence 2.0. 

This third guide focuses on participatory methods of anticipating changing 
technologies and skill demands: technology and skills foresight. We trust the 
practical insights it provides will prove to be useful in your context. 


Jiirgen Siebel Antonio Ranieri 
Executive Director Ad interim head of department 
for skills and labour market 
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CHAPTER 1. 
Technological change and 
skills intelligence 


1.1. Technological change and skill needs 


The impact of technological change on jobs and skills has been at the 
forefront of both popular media and the policy debate, with concerns about 
job automation becoming accentuated as a result of the Covid-19 pandemic. 
Much of this discussion has centred around predictions that almost half of 
all jobs in advanced economies are susceptible to replacement by machine 
learning algorithms (Frey and Osborne, 2013; 2017). Recent studies, that 
have called into question the reliability of such estimates, have noted that 
automation is typically targeted towards the replacement of certain tasks, 
as opposed to occupations as a whole. Accounting for task heterogeneity 
within occupations reveals that the number of occupations at high risk of 
displacement is significantly lower, affecting about 9 to 14% of all jobs in 
advanced economies (Pouliakas, 2018; 2021; Nedelkoska and Quintini, 
2018; Arntz et al., 2017). 

McGuinness and colleagues (2019) also note that the technological 
alarmism that has entered into the policy debate is unwarranted. They show, 
using Cedefop’s European skills and jobs survey data, that only 16% of EU 
employees affected by technological change think their skills will become 
obsolete in the near future and 5% are afraid of technological unemployment. 
Their analysis provides evidence that technological change and innovation 
is not only about automation: it tends to have a positive impact on the task 
content and skill complexity of jobs and hence tends to be associated with 
dynamic upskilling of workers (Deming and Noray, 2020; Acemoglu and 
Restrepo, 2019; Freeman et al., 2020). 

Digitalisation also transforms the nature of the employment relationship, 
for example via the rise of platform work or, more recently, telework (Cedefop, 
2020a). Cedefop (2020b) shows that social dialogue and participatory human 
resource management methods can have a critical role in facilitating non- 
disruptive adoption of new technologies by organisations and workers. 
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Several other macro-trends drive the future demand for skills, notably 
climate change and the green transition, demographic change and migration; 
but looking at how technological progress and innovation impacts skills 
needs is important, as it is widely viewed as the most dynamic megatrend 
shaping the future of work (Brynjolfsson and McAfee, 2014). In the wake of 
the Covid-19 pandemic, policy has also become increasingly concerned with 
(digital) skill gaps affecting workers’ job prospects and the need for stepping 
up investment in lifelong learning to mitigate inequalities due to the ever- 
present, yet growing, digital divide (Cedefop, 2016; Sostero et al., 2020). 


1.2. Skills assessment and anticipation methods 


To understand the extent to which technology is transforming the world of 
work, it is necessary to measure its magnitude and impact on skills demand. 
Labour market and skills intelligence (LMSI, often referred to as skills 
intelligence) provides such information and — provided it is based on sound 
approaches and methods - can serve the needs of those responsible for 
reacting to changing skill needs. 

While analysts and experts have a range of different skills assessment 
and anticipation methods at their disposal, identifying and anticipating the 
pace of technological change in labour markets — particularly in times of rapid 
change - is challenging. With the process of predicting the future becoming 
more complicated and perhaps less certain, the range of methods and tools 
available to those involved in such exercises has become more varied and 
sophisticated. 

Table 1 summarises some of the main methods that can be used to 
gather information on skills needs. Four are particularly important. These are 
those that: 

(a) rely on putting questions to key stakeholders (questionnaire surveys 
of employer and employee skill needs and experience of technological 
change); 

(b) produce quantitative estimates of future skill demands, by extrapolating 
past trends and modelling expected developments; 

(c) source big data on new technologies and skills from a variety of online 
sources (including job portals, CVs, social media, patents, scientific 
databases); 
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Table 1. Tools for carrying out skills assessment and anticipation 


Type of activity Data collected 


Estimates of overall demand and supply of skills and technology use, 


Descriptive statistics/ often based on collating data from various sources (e.g. sector skill 


stock taking studies) 
Quantitative Forecasting or projecting future demand for skills, typically using 
forecasting econometric modelling 
Skills and jobs Assessments of demand for, and supply of, skills and technology use, 
surveys (questionnaire usually with an assessment of the extent to which demand and supply 
surveys) are in balance 

Using matched administrative data sets or surveys to track people 
brite WEE through education and the labour market to see how the former 


influences the latter 


Use of non-quantitative techniques to gauge in-depth information 
Qualitative research about current and future skill demand/supply and technology trends, 
e.g. via company case studies, use of focus groups 


Critical thinking about the future of skills supply/demand and 
technology trends, using participatory methodologies 


Use of web sourcing, combined with text mining and machine learning 
Big data approaches, to collect and classify data about skills, vacancies, 
technologies, etc. 


Foresight 


Source: Cedefop classification. 


(d) use non-quantitative techniques, relying mostly on_ participatory 
stakeholder approaches to gather in-depth information about the state of 
current and future skill demand and supply. 


1.3. Purpose of guide 


This third Cedefop practical guide on understanding the impact of 
technological change on skill demand focuses on participatory methods for 
analysing current and emerging technologies and skill needs: technology 
and skills foresight. 

The goal of technology and skills foresight is to identify a set of major 
areas of technology — usually emerging technologies — which are of particular 
strategic importance to an economy, a region or an organisation and their 
associated implications for skills. Such techniques use various methods 
to collect and bring together data and qualitative information that can 
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provide insight into the impact of technological change on skills demand. 
Opening up the discussion on technology and skills to stakeholders, puts 
them in the driving seat of the data generation process. In this respect, 
skills foresight fundamentally differs from conventional skills assessment 
and anticipation methods (such as skill forecasts, skills surveys, big data 
analyses), where the focus is on producing findings without systematically 
and fully engaging stakeholders in all stages of the work. Participatory 
skills foresight is particularly useful when the focus is on identifying new or 
emerging technologies, such as information technologies, biotechnologies 
and nanotechnologies. 

This third Cedefop ‘how-to’ guide (') builds on the first two on conventional 
and automated skills assessment and anticipation methods. Skills surveys 
and skill forecasts typically rely on methods based on the collection of 
representative labour market information and analysis, using statistically 
robust techniques. Big data and Al-driven analyses apply automated 
knowledge extraction and machine-based techniques to source information 
on technologies and skills from mostly unstructured online sources. 

The guide is structured as follows. Chapter 2 provides an overview 
of the use of participatory methods in skills foresight. The chapters that 
follow review two types of commonly used participatory foresight tools. 
Chapter 3 looks at tools mainly relying on interviews and surveys (including 
Delphi techniques), while Chapter 4 provides an overview of tools collecting 
insight via workshops and other meetings. The guide introduces approaches, 
such as_ brainstorming, STEEPV (social, technological, economic, 
environmental, political and values) and SWOT (strengths, weaknesses, 
opportunities and threats) analysis, scenario development and simulations. 
Chapter 5 reflects on methods for generating policy action plans based 
on foresight results. Chapter 6 concludes with a review of the advantages 
and pitfalls of foresight methods. It also provides reflection on which skills 
anticipation methods are most suited in particular situations and the reasons 
why this is the case. The complementary insight that participatory approaches 
can add to conventional skills analyses and forecasts is considered, and 
the advantages of engaging users in the foresight process are emphasised. 
Throughout the guide, practical illustrations of applying foresight in areas 
related to technological and other changes affecting employment, work and 
skills complement explanation of the methods. 


(‘) This guide is the third of a series. See Cedefop, 2021a and 2021b for the first two guides. 
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Box 1. Cedefop’s ‘how-to’ guides on understanding technological 


change and skills demand 


The purpose of Cedefop’s short ‘how-to’ guides is to provide those charged with 
a responsibility for undertaking skills assessment and anticipation with the means 
to deal with the uncertainty of technological change and its impact on skill needs. 
As the process of predicting the future becomes more complicated and less de- 
terministic, the range of tools available to those involved in skills anticipation has 
become more varied and sophisticated. The Cedefop guides aim to showcase to pol- 
icy-makers and interested analysts how various techniques or methodological tools 
can be readily applied by carefully considering the associated pitfalls and rewards 
of doing so. 

The guides provide targeted information on how interested analysts can adopt and 
implement conventional labour market and skills intelligence methods, such as 
skills surveys and skill forecasts; automated methods reliant on big data and arti- 
ficial intelligence techniques; or technology foresight methods. All can be used to 
detect emerging skill needs related to technological change. Implicit in the guides 
is recognition that no one methodology is likely to provide all the answers and the 
challenge for analysts is to bring together outputs from different approaches to 
skills anticipation. 

The guides build on the existing compendium of guides on skills anticipation pro- 
duced by the ETF, Cedefop and the ILO (2), as well as several previous Cedefop 
reports on skills anticipation methods (°). But they are distinct from previously pub- 
lished methodological handbooks or guides, in that they are explicitly concerned 
with the process of identifying technological (digital) change, a key driver of chang- 
ing skill needs. 


Source: Cedefop. 


() 
() 


See also Sudakov et al., 2016. 
For instance, see Cedefop, 2013; 2015; 2019a; 2019b and Cedefop project Anticipating and 
matching skills. 
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CHAPTER 2. 
Participatory skills 
anticipation methods 


2.1. What is foresight? 


There is much confusion in the policy and academic discourse about what 
the term foresight means. The term was barely used before the 1990s, when 
so-called technology foresight programmes were launched in western Europe 
and elsewhere. These programmes quickly became standard tools for policy- 
making in science, technology and innovation (STI) (See Miles, 2010). 

In these programmes, foresight went beyond the mere anticipation of 
future prospects. It included concepts, such as prudence and preparedness 
for the future. Even with this wider understanding of the concept, foresight was 
criticised. The emphasis on precautionary activities (as in risk management) 
at the expense of actively identifying opportunities (in an entrepreneurial 
approach) was seen as overly restrictive. Despite this, foresight programmes 
in the 1990s were innovative in the sense that they applied many different 
methods, engaged large participant groups and aimed at influencing major 
decisions in STI policy. 

The most successful foresight programmes in the 1990s were different 
from most preceding exercises in three respects (Georghiou et al., 2008). 
They were more prospective (applying methods to appraise long-term 
future challenges and opportunities), more policy-related (strong orientation 
towards informing decision-making via conducting and communicating work 
in relation to policy actors, processes and timetables), and more participatory 
(engaging a wider group of stakeholders and experts). 

Foresight exercises typically go beyond the deskwork that is part of futures 
studies and conventional forecasting and/or modelling in demography, 
economics, climate studies and other areas. The prospective dimensions 
of foresight programmes often use such approaches or draw on the 
evidence they produce. Extrapolations or more sophisticated examinations 
of economic structures and trends, population dynamics, technological 
trajectories and climate change impacts are commonly used. Foresight 
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goes beyond limited planning approaches with restrictive timescales and 
accounting frameworks to consider stakeholder interests, their strategic 
orientations and their possible contribution in achieving policy aims. 

The participatory dimension of foresight goes beyond consulting 
a small stakeholder group and subsequently producing results for wider 
dissemination. In some cases, it involves a public conversation or consultation 
(the latter being more demanding). Substantial involvement of experts from 
a much wider pool is what distinguishes foresight from other forecasting and 
planning activities (‘). 

With the proliferation of technology foresight programmes in the 1990s, 
many practitioners engaged in activities such as technology horizon 
scanning; forecasters started to view their work as being part of foresight. 
This rebranding in recent decades has resulted in the term being used more 
widely to also capture activities that are better considered components of 
more extensive foresight exercises. 

The term fully fledged foresight was introduced to characterise exercises 
combining participatory and other activities to appraise longer-term 
prospects, inform and orient strategies, and ease short- and medium-term 
decisions. As the design of an exercise depends on objectives, context 
and resources, not all foresight is fully fledged. Foresight can take many 
forms, and there is no standard set of tools. What is common to all foresight 
activities is a process involving several steps, which link initial scoping to 
analysis to result dissemination and implementation. 

This guide presents a range of foresight methods, which may be 
implemented in various ways. While many overviews of foresight are 
available (5), the discussion draws on the framework outlined in Miles and 
colleagues (2017). 

Although they have been considered in several foresight exercises, 
skills and occupations are not often the primary interest in STl-oriented 
programmes. There is therefore scope for further development of foresight 
to shed light on the decline/increase in demand for occupations, on the 
emergence of new skills and job tasks (and new skills/tasks combinations) 
and on trends in organising and managing work. 


(*) For an overview of foresight approaches to skills issues, see ETF et al., 2016. The review of 
issues and methods presented here draws on the account presented in Miles et al., 2017, which 
covers STI applications more generally. 

(°°) For instance, Georghiou et al., 2008; Kuosa, 2012; Waverley consultants, 2017. A convenient 
overview of technology forecasting methods is Roper et al., 2011. 
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2.2. The participatory element in foresight 


The participatory dimension of foresight programmes gives a wide range 
of stakeholders the opportunity to influence policy-making. This involves 
going beyond the usual suspects - expert panels and advisors — who in 
the past were commonly consulted in decision-making processes. The 
participatory nature of foresight also builds democratic legitimacy, which is 
seen as particularly important in debates and decisions about potentially 
transformative STI and other key policy domains. 

The wider involvement of stakeholders helps overcome challenges related 
to constraints on public budgets, challenges in understanding the impact 
of new or emerging technologies (for example information technologies, 
biotechnologies, nanotechnologies). Stakeholder involvement in discussions 
on their implementation and their skill implications also helps address 
concerns about the societal impacts and possible by-products of scientific 
and technological progress: the ethics of biomedical applications, the 
employment consequences of automation and artificial intelligence, and the 
health hazards of nuclear power and industrialised agriculture. The potential 
benefits of wide stakeholder participation in terms of foresight inputs and 
outcomes are summarised in Table 2. 

In practice, a foresight exercise often has multiple aims. It may seek to 
achieve a better-grounded and more politically legitimate set of priorities, 
while at the same time exploring opportunities to link different parts of an 
innovation system better. Foresight goals are typically reflected in the design 
of the participatory approaches the exercise employs (°). 

Wide stakeholder participation is possible at different foresight process 
stages. These include scoping (what is the exercise to address), literature/ 
material review (what has already been said and done about this issue) and 
trend analysis and horizon scanning (what is liable to happen in this field). 
Subsequent stages explore and appraise alternative prospects, identify 
possible actions (for example recommending priorities) and follow up on 
proposals (for example via involvement in implementation). 


(°) For further discussion see Amanatidou, 2017; Himmrich and Suss, 2017; Saritas et al., 2013. 
Among more general discussions of participatory methods is Slocum, 2003. 
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Table 2. The benefits of wide stakeholder participation in foresight: 
inputs and outcomes 


Impacts on inputs Impacts on outcomes 


Access to a wider range of technical 
knowledge and professional expertise 
than is possessed by any single person 
or organisation 


Greater legitimacy from a process that has drawn upon 
a broad range of stakeholders and taken account of 
points of view other than those of insiders 


Easier identification of information Establishing new relationships between participants, 
relevant to the goals of the foresight potentially linking together players and enabling more 
exercise progress in different fields of STI 


Better understanding of the strategic 
orientation of stakeholders, and 
their likely responses to possible 
contingencies: this may reveal both 
Opposing and supporting factors that 
will shape patterns of development 


More creative thinking achieved by 
bringing together new sets of people 
and encouraging knowledge exchange 
and shared creation 


Participants may articulate their 

expertise in direct relevance to others Engaging members of user organisations (in addition to 
and be informed about the wider that commissioning the exercise) in activities 
implications of the issue of interest 


Establishment of a pool of participants with in-depth 
understanding of the process and its results, who can 
apply them in future actions and communication and 
are able to translate the implications of the exercise to 
their own stakeholder communities 


Informing participants as to the topic dynamics and 
empowering them to assess better the implications of 
future developments 


Triggering, or at least increasing the feasibility of, 
subsequent foresight exercises among participants and 
others exposed to them 


Encouraging a more forward-looking and more 
participatory orientation among key decision-makers 


Source: Cedefop. 


In practice, foresight stages often overlap and do not necessarily follow 
a fixed sequence. Every stage in the process can be participatory, involving 
stakeholders and/or experts beyond the core groups who design and run 
the project and disseminate its results. Participants can validate or critique 
results from earlier stages and examine and assess their implications for 
their own organisations. 

Table 3 lists typical stages of a foresight process (Miles et al., 2017) and 
describes possible participatory and other activities for each. 
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Table 3. Participatory activities across stages of a foresight exercise 


Stage/phase of 
foresight process 


Initiation: 

scoping exercise, 
determining main foci, 
intended uses and users 


Examples of forms of 
participatory foresight 


Involvement of experts in 
determining topics for examination, 
consultation and face-to-face 
meetings 


Examples of other 
foresight elements 


Steering group decisions, 
interviews with experts and 
stakeholders, stakeholder 
analysis, identification of key 
participants 


Intelligence: 


literature reviewing, horizon 
scanning, trend analysis, etc. 


Workshops where participants of 
various kinds (members of a single 
organisation or drawn from a range 
of bodies) seek to identify main 
drivers of change; 

Delphi-type survey work 


Literature review, 
algorithmic analysis of 
bodies of material, such 

as news sources, patents, 
academic literature, official 
statistics 


Imagination: 

establishing understanding 
of linkages between issues 
and using creativity to 
explore implications 


Discussion and assessment of 
effects of drivers of change and of 
trend impacts in workshop settings 


Statistical analysis of 
relationships in data sets 
and big data analytics of less 
structured material; cross- 
impact analyses (involving 


feedback by experts) 
Integration: : ; 
development and appraisal oat workshops; Computer modelling 
s aming 
of possible futures 
Roadmapping workshops, 
Interpretation: strategy workshops, success- 


identification of strategies 
and prioritisation of actions 


Intervention: 
discussion of proposed 


strategies, priorities and next 


steps with key actors 


scenario workshops; 
Delphi-type policy and goals 
surveys 


Dissemination activities ranging 
from public conferences to 
seminars within stakeholder 
organisations 


Cost-benefit analysis, 
optimisation in models 


Producing reports, priority 
lists, etc. 


Impact: 


evaluation of impact of work, 
follow-up activities specified 


Engaging stakeholders 
(researchers, educators, businesses 
and others) in developing follow- 
up plans, designing research 
projects and teaching programmes, 
developing competitions to support 
innovative activities 


Evaluation projects by expert 
teams 


NB: Material obtained from participatory activities may be fed into non-participatory activities. For example, 
judgements made in workshops or surveys may be used to calibrate a statistical model’s parameters and 
relationships. Outputs of non-participatory activities may be fed into participatory events; for example 
a computer model may be used to give workshop participants immediate estimates of the results of 
their choices. See Giaoutzi and Sapio (2013) for a discussion on interrelating qualitative and quantitative 
approaches in foresight exercises. 


Source: Cedefop. 
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Participatory and non-participatory methods at any stage of the 
foresight process can be closely related. The foresight carried out by Nesta 
and colleagues (2017) explored the future of skills to 2030. Expert and 
stakeholder groups were asked to discuss major trends, and then group 
members were asked to forecast prospects for specific occupations. After 
discussion, where individuals had the opportunity to revise their forecasts, 
the workshop results were fed into a machine learning system, which used 
an algorithm to model how skills and other factors drive these occupational 
prospects. This information was used to select additional occupations, 
which were referred back to the group and, after another discussion round, 
the group work was again fed into the system (’). After several rounds, the 
system was capable of producing expert judgements and forecast trends 
for a set of occupations going far beyond those that had been considered 
in the initial rounds. The team conducting the study subsequently compared 
the results with other studies. 

The extent of participatory activities in foresight may vary, in terms 
of activities carried out over the course of a project, time and resources 
spent, and participant numbers (from tens to potentially several thousands) 
(°). Although in-depth exploration of a particular topic may take months 
or years to complete, in a context of scarce resources and with a need to 
get policy debates underway, it is an option to organise a foresight activity 
requiring relatively less time and resources. This typically involves less effort 
in organising a literature review, eliciting expert opinions and organising 
consultation meetings. With practical constraints often making it challenging 
to carry out all foresight phases in a complete or ideal way, it is often 
necessary to rely heavily on readily available research and analysis. 

Participatory activities also vary in terms of how intensive they are. Some 
activities are little more than gathering feedback on presentations, some 
solicit information and opinions, while others involve substantial dialogue 
which plays a crucial role in shaping the content and structure of results. 
Typically, more extensive activities are less intensive ones, and vice versa, 
although use of modern information and communication systems and tools 
can weaken this link somewhat (Table 4). 


() Avisual representation of this process is Human and machine intelligence hand-in-hand. 
(8) Popper and colleagues (2007) mapped the scale and variety of foresight activities undertaken in 
the early years of this century. 
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Limited participation in scoping, horizon-scanning and planning is the norm 
in many conventional business and government decision-making processes 
that make no claim to constitute foresight exercises. In many activities that 
have been described as foresight (but not the ‘fully fledged’ kind), consultants, 
academic groups, and in-house planning departments rely almost exclusively 
on non-participatory approaches, such as literature reviews, statistical trend 
analysis or computer modelling or simulations. The so-called BOGSAT (bunch 
of guys sat around a table) model can also be found in many organisations, 
where, for example, applied research and development (R&D) decisions are 
at least in part made by a panel relying on information from its academic and 
business members and expert interviews (°). 


Table 4. Typology of foresight activities by process duration, 
participant engagement and consultation scale 


Brief process duration Lengthy process duration 


Few Many Few Many 
participants participants participants participants 
interviews soliciting 
reviewing project conventional and collating 
Low participant outputs; surveys; occasional 
engagement conventional public reactions to 
seminars with consultations material posted 
Q&A online 
scenario 
workshops; 
roadmapping 
High workshops consensus 
participant preparing Delphi surveys conferences; online gaming 
engagement proposals for citizen juries 
implementation 
and follow-up 
activity 


Source: Cedefop. 


(°°) For example, in the UK, applied R&D decisions are at least in part made through a panel that 
takes into account information from academic and business members, and from irregular 
reviews of Technology Innovation Futures largely drawn from interviewing key experts; see 
https://www.gov.uk/government/collections/technology-and-innovation-futures 
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This guide focuses on the more intensive methods. It concentrates on 
participatory elements in the generation and appraisal of future prospects 
and developing recommendations for action. It is acknowledged throughout 
that available resources (what technical skills for conducting foresight 
are available, what funds are available for organising meetings, need for 
consultancy work) and the context in which foresight takes place determines 
which participatory methods are suitable. Examples of context are openness 
of a culture to consultation and dialogue, the extent to which views are 
polarised, and the availability of solid evidence on the topic. 


2.3. Recruiting foresight participants 


One category of participants involved in foresight exercises is experts. 
Sometimes these have foresight expertise, as is the case with futurists. As 
well as providing advice on process design, they can also use their expertise 
to be provocative in workshop discussions, suggest possible wild cards to 
stimulate out-of-the box thinking and reflect on the wider implications of the 
issue at hand. 

Domain experts — knowledgeable members of research or practitioner 
communities with relevant experience — are more commonly involved. They 
should be recognised as experts by other members in their professional 
community and be willing and able to cooperate with experts in other fields. 
It is desirable that they participate based on their own knowledge rather than 
as representatives of a body, as this limits their marge-de-manoeuvre and 
creative thinking. 

Domain experts are often identified from their publications and 
presentations, recommendations from professional or other relevant 
associations, or via personal networks. It is good practice to maximise 
diversity among the invited expertise, by involving promising younger 
professionals and underrepresented groups; these often have unique insights 
into emergent issues others may lack. 

Stakeholders form a second category of foresight participant. Stakeholder 
analysis is typically used to identify key organisations and social groups that 
may be affected by the issue at hand and by recommendations emerging 
from the foresight activity. Some organisations may be selected because 
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they play an important role in ensuring the feasibility and implementation of 
the foresight process and its results (1°). 

Where there are many broad stakeholder groups (for example blue-collar 
workers, small business owners, young people), it may be appropriate to 
involve participants who have a good overview of these groups and the 
issues they are confronted with, such as trade unions and professional 
association representatives and voluntary association leaders. It is crucial to 
involve participants who are not constrained to voicing the official positions 
of the organisation they represent. 

Foresight is intended to inform action. If the exercise is simply box-ticking 
(for instance, to provide proof a foresight was undertaken), its effects are likely 
to be limited. Institutions or bodies commissioning foresight often do so in the 
context of their own strategy development. To ensure messages are really 
accepted and set the grounds for further action, it is particularly important 
they engage their members. Foresight particularly benefits from a ‘champion’, 
who fully understands the process and underlying reasons and rationale for 
the results achieved, as well as being able to articulate their relevance. 

The number of participants to be engaged depends on the scale of the 
exercise. While a typical workshop might involve 20 participants, much larger 
groups can be accommodated. This requires effective management, which 
can entail, for example, splitting a large group into subgroups. A large-scale 
survey or consultation might involve thousands of participants. 

Recruiting and motivating participants can be challenging. The more input 
is expected, and the more extensive the other commitments of the people 
to be enlisted, the greater difficulties can be. A major intrinsic incentive for 
participants is the belief that they are contributing in a meaningful way to 
a process that will affect an issue they care about and/or are engaged with 
regularly (for instance, as part of their work). 

A request to participate in a survey is often persuasive only for a small 
share of those invited. Personalisation of the request and statements about 
importance by a credible authority are often used to boost participation. 
Willingness to participate also depends on the quality of survey design, the 
ease of the task and the number of questions. Taking part in a workshop 
requires time commitment and may be daunting to those unfamiliar with this 
type of practice. It is important to explain clearly what the exercise entails and 


(*°) Many tools and frameworks for conducting a stakeholder analysis (stakeholder mapping tools) 
are available online. 
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to point out how participants themselves may benefit. In conclusion, several 
factors jointly determine success in recruiting foresight participants (Table 5). 


Table 5. Factors influencing foresight participant recruitment 


|Factor = Crucial questions/issues 


Clarity of requests 
to engage with the 
exercise 


How well is the exercise explained or expectations for the participant 
outlined? What contribution is expected? 


Customisation of 
requests 


Extent to which the requests are tailored to the person addressed, rather 
than being clearly part of a widespread ‘trawling’ exercise. The latter strategy 
runs the risk that recipients will have the impression other participants may 
be representative of people with spare time rather than experts. 


Expected impacts 
of the exercise 


Is the activity purely academic, or does it have to influence the policy 
discourse or management decisions? What opportunities are foreseen to 
comment on process and outcomes? Does the exercise have the support of 
leaders of important organisations? 


Personal benefits 


What might the participant gain by way of knowledge, early warnings 
(including early access to results), social contacts, prestige? Will expenses 
of attending meetings be covered? 


Required effort 


Source: Cedefop. 


Roughly how much time is required? Over what period? How flexible are 
these arrangements? 
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Many technology foresight exercises aim at identifying major technologies 
— usually emerging technologies — which are of strategic importance to 
economies or organisations. Mixed method approaches, combining non- 
participatory techniques (for example, literature reviews, bibliometrics 
or patent analysis, web-searching) and participatory methods (surveys, 
workshops), are commonly used. Often non-participatory tools are used 
to identify and group areas of technology, while participatory ones are 
employed to assess their significance in terms of competitiveness, economic 
growth, environmental quality, health and safety in the workplace and other 
potential outcomes. 

The result will generally be a list of technologies (or areas), outlining 
their perceived importance, and other elements such as urgency and 
risks. To assess capabilities and risks associated with various technology 
fields, surveys, interviews, and/or panel workshops are often used. Such 
participatory techniques are also well-suited to identifying implications for 
action. 

Such critical technologies foresight analysis should also consider other 
types of innovation, including social innovation. Critical technologies 
foresight studies have been criticised because they prioritise thinking 
about technological opportunities rather than considering the scope for 
broader social or economic interventions. An example of such a broader 
approach is identifying critical skills a society needs to invest in to ensure 
that the implementation of existing and new technologies is compatible with 
environmental sustainability ambitions (""). 

Expertise on key aspects of the subject matter is the backbone of insightful 
foresight. Some experts may contribute in a non-participatory mode, for 
example by providing literature reviews or calibrating computer simulations. 
Others will participate more broadly, for instance via interviews and surveys. 


(1) For instance, it should be possible to estimate the required numbers of people with skills 
relevant to establishing solar or wind-based power generation facilities, retrofitting buildings with 
more effective insulation or natural ventilation systems and so on. 
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Such methods are particularly appropriate for engaging technical experts 
and well-informed stakeholders. 


3.1. Horizon scanning: interviews and opinion 
surveys 


Surveys gathering expert opinions on important developments in their areas 
of expertise may be carried out by post, telephone, online or face to face. 
Such surveys can be used to shape Delphi studies or scenario workshops 
(discussed in more detail in Section 3.2 and Chapter 4). 

One approach is asking respondents to provide open-ended answers 
to questions on the major drivers and shapers in their area of expertise. 
This provides opportunities for reflecting on the problems and needs they 
create, solutions and innovations that might be considered, and research, 
knowledge, or capability needed to achieve them. 

This approach makes it possible to engage people beyond the core 
foresight team or panel and benefit from their insights and reflective inputs. 
Careful question design and selection and motivation of respondents are 
drivers of achieving high-quality responses. Sufficient time and expertise 
must be allocated to process and analyse the significant volume of qualitative 
data generated. 

The Australian Commonwealth Scientific and Industrial Research 
Organisation (CSIRO) used interviews to inform foresight on future 
employment, occupations and skills (Hajkowicz et al., 2017). A series of 
‘convergent interviews’ ensured that, as the foresight progressed, the 
questions became more rigorous and detailed. Probe questions were used 
to converge towards shared insight. The opening question was: What are 
the key trends that you can see emerging that will influence the future of 
work over the coming 20 years — so out to 2035? The interviewer abstained 
from contributing content and insight, using other prompts to continue the 
conversation, and sometimes posing questions to validate points made in 
previous interviews. Workshop materials, generated based on a modified 
version of the interview approach, complemented the face-to-face and 
telephone interviews. The result was a list of major trends, which could be 
classified using the STEEPV framework (as described in Chapter 4) and used 
in shaping and developing scenarios. 
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Cedefop’s skills governance reviews, carried out in several EU countries 
(Box 2) (12) used a similar approach. After identifying key insights into success 
factors and bottlenecks in the national skills anticipation and matching 
system, stakeholder interviews helped develop more detailed understanding 
on challenges and opportunities. This was used to develop Delphi-style 
consensus-building exercises (Section 3.2.2) which were carried out to 
shape stakeholder-backed national policy roadmaps (Cedefop, 2020c; 
2020d, 2020e; 2020f). 


3.2. Delphi and other survey methods 


3.2.1. Surveys 

Surveys allow large numbers of people to be involved in providing information 
at their own convenience, without having to attend a meeting. Sometimes, 
workshop or conference participants are invited to complete a survey during 
a meeting, but this usually involves short and simple surveys. 

Recruiting appropriate types of respondent can be challenging. Busy 
people receiving many survey invitations may reject such requests because 
they have no time to participate in all of them. Making a survey attractive and 
meaningful to respondents can encourage participation. Using incentives 
and endorsements from influential and respected figures are other common 
motivation mechanisms. 

Surveys require people to express opinions (for example: how soon is 
it likely that these tasks will be completely automated?), attitudes (how 
important is it to increase equality of opportunity in terms of access to these 
jobs?), or state facts (how many people are employed in this firm?). Use of 
standardised categories and Likert scales to allow respondents to tick boxes 
to indicate level of agreement with statements, or the range that they fall 
within, results in relatively structured data (see Cedefop, 2021a). Although 
such information can be valuable in assessing the extent of viewpoint 
divergence in a community, it does not provide the rich information that can 
be captured using open-ended qualitative methods. 

Nevertheless, structured information can help situate and contextualise 
more detailed information obtained from interviews, site visits and other 
methods. Surveys can also be structured to explore the roots of differences 


('2) See Cedefop’s Anticipating and matching skills project for further details. 
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in viewpoints. They may, for instance, reveal that people in particular age 
cohorts or employed in certain industries or professions think differently 
about the extent or impact of change from others. 


Box 2. Cedefop’s skills governance stakeholder interviews 


As part of Cedefop’s country support initiative to improve EU Member States’ skills 
anticipation and matching systems, stakeholder interviews were carried out in four 
EU countries (Bulgaria, Estonia, Greece and Slovakia) in 2017-18. The stakeholder 
interviews were based on a generic questionnaire which helped develop in-depth 
understanding about the key elements in Cedefop’s skills governance analytical 
framework: 

regulatory framework: the role of organisations in labour market and skills in- 
telligence according to regulation, its practical implementation and suggested 
improvements; 

institutions: stakeholder participation in LMSI, the functioning of bodies of ex- 
change and relevance of external experts; 

LMSI management: collection of labour market information (organisations, organ- 
isation role, stakeholder involvement), own experience with stakeholder collabo- 
ration and evaluation, involvement in the interpretation of results, involvement in 
policy actions, difficulties in engaging with stakeholders, practical collaborations 
of agencies, conflict resolution; 

LMSI tools/methods: methods used in LMSI, suitability of data and methods, suffi- 
ciency of detail collected, suggestions for improvement; 

LMSI dissemination: obtaining LMS! information, targeting LMS! information, LMS! 
customisation, LMS! presentation and dissemination; 

sustainability and reputation: confidence in the existing skills anticipation and 
matching system or practices, key limitations of existing system, view on neces- 
sary future development, planned developments. 


A wide range of mostly open-ended questions on each topic was asked during 
semi-structured face-to-face interviews. To obtain a minimum degree of generalis- 
ability based on the findings, findings were aggregated in analysis by type of stake- 
holder. Representativeness was ensured by confirming that opinions about particular 
issues were shared and possible development opportunities widely supported (or at 
least not refuted) by most participants or at least by a critical subgroup of the most 
important stakeholders. 


Source: Cedefop, 2020c; 2020d; 2020e; 2020f. 
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The information technology revolution study (Rush and Miles, 1989) 
used a survey to examine varying viewpoints in a population. Around 100 
experts were asked to assess the extent to which a range of technological 
trends would develop in the next two decades. Issues covered included 
occupational polarisation, teleworking, and the proportion of the workforce 
required to undergo two or more phases of retraining in their career. 

After identifying a wide diversity of viewpoints, statistical analysis 
revealed that respondents’ expectations largely varied in terms of two 
underlying dimensions: how fast change would happen and whether its 
impact on the working population was broadly positive or negative. These 
dimensions were used to construct four scenarios, based on the average 
expectations of outcomes. 

In studies conducted by Nesta and colleagues (2017), participants in 
workshops were invited to complete surveys. Two workshops were held 
that brought together 12-13 experts from industry, government, academia 
and sectoral representatives in the UK and USA. Participants received 
extensive information on major trends, including those concerning 
population ageing, industry 4.0 and_ sustainability/climate change. 
A session then used creativity techniques to stimulate participants’ thinking 
about possible implications for existing and new jobs. Following this, they 
received information on employment in 10 occupations: job titles, tasks 
and skills required, and data on employment trends and top industries in 
terms of employment in the occupations. 

Participants used an online system to rate occupational prospects (for 
2030). They were asked two questions which used three-point scales. The 
questions were: 

(a) what will happen to the share of total employment held by this occupation 

(higher share, same share, or lower share)? 

(b) what will happen to the number of people employed in this occupation 

(grow, no change, or decline)? 


They also indicated how certain they were about their judgements and 
were given the opportunity to contribute in writing anything they considered 
necessary to qualify these judgements. 

During the group discussions that followed, people could change their 
responses. The expert judgements the workshops helped shape were used 
in a machine learning algorithm, which was later used to estimate impacts 
for many more occupations and for 120 typical skills associated with them. 
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Combining sophisticated participatory and non-participatory techniques led 
to results that are less apocalyptic about the employment impact of new 
technologies than those produced by many other analysts ('°). 

By requesting more open-ended input than is common, some surveys 
mimic the information collection process of interviews. As part of the 1995 
UK technology foresight exercise, expert respondents were requested to 
describe the most important trends in their area of expertise, problems 
encountered and technological innovations that might either help overcome 
them or lead to new challenges. The information was used in the development 
of Delphi surveys during follow-up stages of the work. 

Demanding questionnaires run the risk of achieving low response rates 
and need a lot of effort to encourage participation. While open-ended surveys 
can give respondents more time to reflect on their answers and lead to more 
insightful information, this needs to weigh against the risk of them being set 
aside or not (fully) completed due to survey fatigue. 

It is for these reasons that open-ended questions tend to be more 
appropriate for interviews and group discussions in workshops (see also 
Chapter 4). Workshops also give participants the chance to be inspired 
by each other’s views. One of the advantages of survey methods is that 
they reduce the risk of some respondents exerting too much influence 
on the discussion, because of seniority, personality or presentation style. 
Delphi-style methods can be a good compromise between encouraging 
in-depth exchange of views among a pool of respondents, while avoiding 
disproportionate influence of one or more of them. 


3.2.2. Delphi 

Delphi surveys involve more intensive expert or stakeholder participation, not 
only because of the possibility of open-ended questions but also because 
they typically use multiple iterative rounds. This critical feature which makes it 
possible to construct feedback loops, also gives respondents the opportunity 
to reflect on their own views in light of other participants’ contributions. 
Typically, this involves questions probing whether participants wish to modify 
their responses. A less used approach is to formulate questions considering 
responses to questions posed earlier. 


(8) For a readable summary of the workshops (and some of their results), see: 
https://medium.com/@wendyinfutures/forecasting-jobs-and-skills-2030-98d26008f808 
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There are many types of Delphi survey ('4) and practice is evolving as 
more online tools become available. With modern Delphi tools, results can 
be updated in real time. The method can be applied to elicit opinions, such 
as when a certain development will occur. It can also be applied to set goals 
(which skills should be prioritised in large-scale training programmes?) or 
assess the impact of various courses of action (how does it impact sectoral 
growth or employment levels?). Anything that can be presented in the form of 
a concise and unambiguous question, that can be answered with sufficient 
degree of precision, can be a subject in a Delphi survey. 

By far the most familiar application of Delphi is forecasting. Typical core 
questions concern the anticipated time horizon of a specific development ("5). 
Alternatively, questions can probe the extent to which a particular development 
will have materialised by a given date ('§). Formulating such core questions 
requires considerable effort. Survey designers may be inspired by earlier surveys. 

In addition to forecasts, many Delphi surveys ask respondents to provide 
other information. This can include questions on factors that might inhibit 
or accelerate developments, such as cost, technical difficulties, ethical 
issues, resistance from consumers/workers/managers and skill shortages. 
Questions may also be used to help participants reflect on possible impacts 
of changes: for instance, whether the development would lead to job losses 
or gains, upgrading or deskilling of work, more or fewer opportunities for 
specific categories of employees (Such as people with physical disabilities). 
Respondents may also be requested to provide opinions on whether 
developments increased or decreased demand for particular skills sets 
(physical dexterity, numeracy, interpersonal communication skills). It would 
be unwise to treat the answers to such queries as conclusive and complete 
assessments about such impacts. They do, however, shed light on the 
degree of consensus or disagreement among experts. 


('4) The classic overview of the Delphi method is Linstone and Turoff (1975). 

(5) An example question is: at which of these dates would you first expect a majority of the 
workforce to be continuously providing data on their location, tasks, breaks, etc. by means of 
mobile communications? 

(6) For example, in 2025, what share of construction workers will be working alongside robots in the 
erection of office buildings in cities? 
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Box 3. Eurofound’s Euforia project: Delphi exercise example 


The Euforia project (Loveridge et al., 2004) was carried out to explore the implica- 
tions of the development of the knowledge society for working and living conditions. 
The project was undertaken in three countries. A total of 32 statements were gener- 
ated on topics related to the knowledge society. A Delphi survey was then conduct- 
ed. Significant problems (for example translation issues, computer scripting) had to 
be overcome, as is typically the case in cross-country studies. 

A key question was whether respondents found the statements to be a plausible 
characterisation of the development of the knowledge society in their own country. 
Table 6 features examples of statements that were found to be ‘about right’, meaning 
reasonable characterisations. Additional questions focused on soliciting opinions on 
whether the development would have a positive, negative or neutral impact on each 
of the main features of concern, should they be realised. 

Rich and extensive results were obtained, with national differences and interesting 
viewpoints about topics seen as overstatements. For instance, among topics with 
major impacts, lifelong learning was believed to improve all three industrial relations 
factors (employer-employee relations, economic growth/wealth creation and entre- 
preneurship and innovativeness), improve employee autonomy and responsibility, 
and create more jobs and strengthen social cohesion. There is uncertainty in the type 
of impact that lifelong learning will have on work-life balance, social exclusion and 
disagreement concerning sustainability and environmental quality. 


Source: Loveridge et al., 2004. 


Box 4. Cedefop’s skills governance country reviews: an example of 


Delphi exercises 


As part of its skills governance country support initiative, and following the completion 
of the stakeholder interview phase (Box 2), Cedefop used an online Delphi method to 
carry out consensus-building exercises in four EU countries. A first step was to secure 
the involvement of main national institutions and stakeholders involved in the skill an- 
ticipation and matching system. Three sequential rounds (Table 7) were then carried out 
using questionnaires in the respective language, with several open and closed questions 
administered online. Participants received extensive guidelines and were asked to com- 
plete the questionnaires on their behalf and the institution they represented. Each round 
lasted between two to three weeks. Full confidentiality and anonymity were guaranteed. 
A summary of results from each round was presented before each subsequent round. 


Source: Cedefop. 
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of topic statements seen as 


‘about right’ for characterising 2015 and views as to their 


impact 


Topic category Topic statement 


A major increase occurs in 
my country in the use of 


Impact assessment 


Social cohesion: ?? 
Social exclusion: ?? 
Environmental quality: + 


13: electronic networks for remote | Industrial relations: ?? 
Industrial supervision of new kinds of Growth/wealth creation: + 
relations work (teleworking, mobile Entrepreneurship/innovativeness: + 
working), and new atypical Employee autonomy/responsibility: + 
forms of work. Work-life balance: ?? 
Job creation: + 
Social cohesion: + 
Harmonisation of educational | S0c'al exclusion: - 
2 : Environmental quality: +? 
standards.\ineluing Industrial relations: +? 
18: certification) across eae 


the EU increases trust 
and transparency in my 
country’s educational system. 


Living conditions 


Growth/wealth creation: + 
Entrepreneurship/innovativeness: +? 
Employee autonomy/responsibility: 0? 
Work-life balance: 0 

Job creation: + 


Europe has developed into 
24: a leading force in the area 
Sustainability and of sustainable development 
development and the use of environmental 
technologies. 


Social cohesion: + 

Social exclusion: -? 

Environmental quality: ++ 

Industrial relations: 0? 

Growth/wealth creation: + 
Entrepreneurship/innovativeness: + 
Employee autonomy/responsibility: 0? 
Work-life balance: 0? 

Job creation: + 


NB: ++: strongly increases 
+: increases 
0: no effect 
-: decreases 
?: some uncertainty (little consensus) 
??: very uncertain (bimodal distribution). 
Source: Loveridge et al., 2004. 
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Table 7. Delphi rounds carried out as part of Cedefop’s skills 
governance country reviews 


Round 1: Prioritisation 


An online questionnaire was developed outlining a number of potential focus areas related to the 
main areas of interest for each country's skill governance review. Respondents were asked to 
prioritise these and clarify their positions (i.e. why was an issue selected; main challenge to be 
resolved; suggestions for improvement). 


Round 2: Deepening of stakeholder positions 


This round focused on the priority issues of stage one and sought to work towards a shared 
problem analysis. Possibilities were explored with the aim to agree on possible feasible solutions, 
to identify steps required to implement them, and to explore how different institutional players can 
contribute towards common solutions. 


Round 3: Achieving consensus 


The final round aimed at defining follow-up steps and setting a concrete timetable for the actions 
stakeholders reached consensus on. 


Source: Cedefop. 


In early Delphi practice, respondents were typically asked to indicate why 
they were expressing particular points of view, possibly prompted by having 
recently seen a newspaper article or project findings. Although extracting 
such open-ended information is relatively uncommon nowadays, due to 
the additional load posed on respondent memory and the subjectiveness 
of assessments, online techniques have made it possible to acquire rapid 
insight into extreme responses and feed this back to the group. What is 
common is to ask respondents to rate their expertise, or their familiarity with 
a particular field. It is then possible to compare results between more and 
less expert respondents. 

Designing and administrating Delphi studies is time-consuming and labour 
intensive, but they contribute to more thorough analysis and insightful results. 
Findings generally lend themselves well to graphical representation (for 
example bar charts, infographics) of expectations regarding the relative pace 
of development of different technologies in different applications, impacts on 
skills and working conditions and others. One of the main problems of visual 
images based on Delphi is they tend to be overly persuasive: they cannot 
transfer the complexity and varied argumentation and reasoning present 
in complete qualitative results, such as the proceedings and outcomes of 
expert workshops. 
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Box 5. Delphi focused on critical technologies and emerging skills 


Ahlqvist (2003) presents an example of an exercise combining Delphi, critical tech- 
nologies and other approaches to explore expert opinions on emerging skill and 
occupational needs. Having established the importance of information technology, 
biotechnology, and nanotechnology/materials technology, major trajectories and 
emerging areas in each were assessed using a survey. These were then rated, in 
terms of their likely introduction in the medium-term future (at the time, by 2015), 
with leading examples: targeted medicines; sensors (for example monitoring haz- 
ardous changes in environmental conditions); biomedical, photonic and intelligent 
(self-repairing) materials; and 3G and virtual reality systems (which can be used to 
enable distance working and delivery of healthcare and other services). 

Given these developments, respondents were then asked to assess the employ- 
ment impact in 12 occupations, ranging from architects and biochemists/biologists 
to office and manufacturing blue-collar workers. The respondents were also asked 
to reflect on new professions that might emerge. Suggestions ranged from artificial 
organ designer, Al consultant and gene therapy consultant, to smart home designer 
and social network analyst. 

In the final step, ratings were elicited as to how far these suggestions were plausible, 
and, if so, when they would become reality. Most respondents viewed all new pro- 
fessions proposed as plausible reality in the next 10 years. Artificial organ designer, 
Al consultant and gene therapy consultant were among the slowest to be realised. 
Social network analysts were seen as most rapidly maturing towards a well-defined 
profession. Other roles seen as rapidly emerging included cool consultant (providing 
advice on fashion and changing tastes), virtual doctor, geo-informationist, and cy- 
brarian (specialising in monitoring and locating online resources). 


Source: Ahlqvist, 2003. 


3.2.3. Cross-impact analysis 

A weakness of many Delphi-type surveys is that the questionnaire attempts 
to collect views on a range of topics, with little or no exploration of how 
these might be related. Survey responses can always be examined to see 
the statistical relationships between expectations about different topics, but 
this approach is imprecise because it does not provide full understanding of 
relationships in a system. Cross-impact analysis can be used to deal with 
this problem. It involves going beyond requesting respondents to rate the 
likelinood of various events occurring by a point in time, questions usually 
addressed in a Delphi or similar survey. Cross-impact analysis is possible 
by asking participants to indicate the relationship between different events, 
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for instance how likely is it that event A will happen if event B happens? 
And how likely is it that event A will happen if event B does not happen (17)? 
The result is a matrix depicting interrelationships between sets of events, 
which can be seen as systemic representation of the relationships that are 
explored. 

The resulting information can be used in several ways. Some events 
may turn out to be critical in terms of their impact on others. Cross-impact 
method practitioners frequently derive a set of basic scenarios, representing 
every possible combination of occurrence/non-occurrence of each event, 
with probabilities assigned to each ('8). But such information is based on 
many difficult judgements about combinations of events participants have 
to make, which rise rapidly as the number of events increases. In practice, 
few events can be considered without overloading respondents, which is 
a serious limitation of the approach. Unless a small number of critical events 
can be sensibly identified, the method is of limited practical use. Other 
approaches are needed to examine the wider implications of events and the 
factors driving them. 


('”) Usually, these probability ratings are made on simple scales, for instance a five-point scale 
where 1=very low probability and 5=extremely high probability. 

('8) The standard way of calculating the probability of each scenario is via some form of Monte Carlo 
simulation, taking the cross-impact matrix as a computer model which is run repeatedly, so that 
a sample of scenarios is produced. See Godet,1975. 
Also see Banuls and Turoff (2011); Scapolo and Miles (2006) for descriptions of the literature 
combining and contrasting cross-impact and Delphi approaches. 
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4.1. Ground rules and good practice 


Workshops are used in many foresight exercises, to facilitate and provide 
a structure for face-to-face encounters between participants. Physical 
presence is particularly effective in terms of engaging people in participatory 
dialogue and in achieving collective learning outcomes. 


4.1.1. Designing effective workshops and follow-up 

As ensuring wide participation in discussions in larger groups is difficult, 
larger foresight workshops or conferences alternate plenary activities and 
small-scale workshops. Provided they are well-facilitated, such break- 
out sessions can be effective means of exchanging views on typical skills 
and technology foresight issues, such as emerging technologies, drivers, 
scenarios, professions and skills. Reporting back workshop deliberations 
in plenary is common, and in the context of foresight such reports often 
aim at providing a convincing account of collective understanding or beliefs 
(‘selling’ results). 

Where case workshop outcomes prove surprising or contentious, 
clarification requests are likely and a debate may follow. To gain insight into 
what larger audiences think about the outcomes of small group discussions, 
voting may be used. Alternatively, workshop findings may be taken as they 
come, and used to inform follow-up activities, such as constructing scenarios 
based on identified key drivers of change. 

In larger foresight exercises, it is common for workshops or break-out 
groups to have a scribe tasked with note-taking and a designated rapporteur 
who reports findings and ideas to the plenary. 

As well as a report summarising a foresight workshop for participants 
and/or the sponsor(s) of the exercise, additional structured analysis can be 
provided. Mind maps - visual representations of links between concepts 
and ideas — are a useful tool. They usually start from a central idea which is 
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then linked to other main ideas; these linkages may reflect the relationships 
between concepts that emerged in the workshop discussion ('°). 


4.1.2. Workshops: do’s and don’ts 

Workshop participants should be given the opportunity and encouraged to 
state their point of view, and treated with respect when doing so. The fact 
that some participants may be more expert on specific topics than others 
does not mean that they should be dominant in all matters. 

A good facilitator helps participants voice their own views and share 
their knowledge while treating others with different viewpoints with respect. 
Facilitation requires a wide spectrum of social and interpersonal skills, ability 
to use support tools effectively (for example flip charts/post-its or their 
digital equivalents), and, ideally, also some prior experience on the topics 
discussed. Skilled facilitation involves dealing with conflicting opinions, 
encouraging less articulate and more reserved group members to voice their 
views and concerns, providing space for expressing minority or contentious 
views, and controlling the more dominant and less well-mannered ones. 

Capturing ideas as they are presented is useful, because this helps 
establish a record of the discussions that may be used for follow-up stages 
or report preparation. The approach also helps transmit the message to 
participants that their inputs are valued and noted and makes it possible to 
refresh the collective memory and quickly return to points made earlier. It 
can be helpful to have a space where ideas (for example wild cards) can be 
‘parked’ for later discussion. 

A facilitator who is fully aware of the main do’s and don’ts (Table 8) 
contributes greatly to the effectiveness of workshops and to the quality of 
the results they can achieve. 


4.2. Decision-making 


In many workshops and similar events, decisions must be made. Typically, 
not all generated can be fully considered. While selection may be necessary, 
it is important to retain ideas that had to be put aside. Recording them, 


('8) On historical antecedents to mind maps, see: https://www.mind-mapping.org/blog/roots-of- 
visual-mapping/. There are many software products available to support mind mapping. For 
a list see: https://www.mind-mapping.org/index.php?title=Main_Page 
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Table 8. Workshops in foresight: do’s and don’ts 


Deviate from the Chatham house rule — the 
principle that participants are free to use the 
information received, but neither the identity nor 
the affiliation of the speaker(s), nor that of any 
other participant, may be revealed 


Explain how the insight and materials 
generated will be used (including confidentiality 
of personal information) and address any 
disagreement beforehand 


Explain ground rules for open communication at | Leave room to believe potentially disruptive or 
an early stage uncivil behaviours are acceptable 


Allow participants to criticise openly and put 
down others 


Promote a climate of respect and trust 


Encourage all participants to express their view | Give some participants or experts opportunities 


freely to dominate the discussion 
Allow participants to challenge ideas and Allow combative discussion or interaction styles 
disagree politely (e.g. dismissive, sneering responses) 


Promote creativity by stimulating out-of-the-box | Engage in discussion or debates before 
thinking and valuing every idea or suggestion a Critical mass of ideas has been collected 


Source: Cedefop. 


for possible presentation in workshop documents or progress reports, will 
make participants who developed them feel acknowledged. Almost all ideas, 
including non-short-listed ones, are useful to follow-up work. The exception 
is ideas that cannot be retained because, for example, they are factually 
inaccurate or seriously flawed or unacceptable in ethical terms. What to 
retain or not should be a conscious process. Sometimes ideas that appear 
unrealistic at first sight may turn out to be plausible after all. Moral objections 
can be based on misapprehensions. 

In situations where participants overwhelmingly favour or reject an idea, 
group decisions can easily be made. With many ideas on the table, this is 
less likely, and voting is commonly used to aid decision-making. Polls can 
ask participants to nominate, for instance, the three most important factors 
driving employment or skill demand, within a given time horizon. They can 
also be used to ask participants to reflect on the likelihood that particular 
technology trends will take off or if there are barriers (for instance, skill 
shortages) to their wider implementation. 

Opportunity for dialogue and exchanging views is a key advantage of 
workshops. They allow participants to gain insight into the underpinnings 
of assumptions and points of view. When voting is seen as stifling views, 
especially among participants with deep expertise or strong feelings about 
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specific issues, the added value of workshops over questionnaire-based 
surveys may be diminished. On the other hand, voting can also be a way 
to avoid highly articulate individuals (or worse, those with a bullying or 
dismissive attitude towards others) hijacking group discussions. 

Opportunities to use polling are numerous and rapidly growing with 
the proliferation of digital tools for laptops, tablets and smartphones. 
Such tools incorporate sophisticated rating scales and innovative ways of 
expressing relative preferences. More conventional tools often involve giving 
participants several votes to be used to indicate preferences (perhaps using 
post-it notes, stickers, markers). They may use lists of ideas on thematic flip 
charts covering the social, technological, economic, environmental, political 
and values factors often considered in foresight exercises (see Section 4.4). 
Presenting options and/or alternatives in terms of technologies, professions, 
scenarios or policy actors is another widely used approach. 

Colours may be used to characterise specific aspects, choices or 
assessments, such as importance, positive/negative impact, and highly 
probable/uncertain. Voting set-up can be tailored to different circumstances. 
This concerns the number of votes allowed, whether it is possible to cast 
multiple votes for one idea, and whether participants are discouraged from 
voting on their own or group ideas. Other set-up decisions relate to whether 
participants are asked to distribute votes equally across several flip charts, 
whether it is mandatory to use equal numbers of votes for each colour 
in a three-colour system and others. There is little guidance as to which 
approaches work best. Experienced facilitators will often sense whether 
using more complicated approaches has benefits. 

Unless IT-based solutions are employed, the voting methods described in 
Sections 4.1 and 4.2 have several drawbacks. They do not allow participants 
to vote anonymously and — because earlier votes are visible to those voting 
later — results may incorporate a degree of social influence. 

Following the vote(s), the facilitator must point out which ideas have 
greater support and request the group to focus on them. When several ideas 
have similar levels of support, they may be connected. In other situations, it 
is more appropriate to ask the group to help in making a more definite choice 
(for example, by show of hands after a brief discussion). The group should 
always be reminded about the uncertainties or implications of the ideas that 
ultimately were not selected, particularly those that received substantial 
initial endorsement. 
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4.3. Brainstorming 


Brainstorming is used to unleash creativity in a group and to ease generating 
new viewpoints. The philosophy of brainstorming is allowing participants 
to present ideas freely without being influenced by comments or value 
judgements by others. Ideas are evaluated after brainstorming, when 
a sufficient number and diversity have been contributed. Participants must 
have equal status and equal opportunities to share their ideas and this 
principle has to be clearly communicated at the outset. Reminders may be 
needed during the brainstorm. 

After explaining the rules of the process, the facilitator explains the 
brainstorm topic to be addressed. This could entail posing questions, such as: 
(a) what are the main factors driving or inhibiting the introduction of new 

technologies in one or more professions or workplaces? 

(b) what skills will be required if these technologies are to be adopted for 
particular applications? 
(c) what risks and wild cards might be encountered? 


It is important to be clear about the focus of question(s). If the interest is 
in the impact of new technologies on future skills demand, two brainstorms 
might be needed. The first would be about identifying factors that could 
influence the pace of adoption of these new technologies; this step would 
require clarity about the definition of the technology. The considerable 
ambiguity in terms like artificial intelligence and robotics or digitalisation in 
practice shows how important this step is. It is also important to be clear 
about time horizons, as trends and prospects five years ahead differ markedly 
from those looking 20 years into the future. After factors influencing the pace 
of technological change have been identified, in the second brainstorm 
the focus can be on the implications of adopting technologies for work 
organisation, jobs and skills. 

In brainstorming sessions, participants are asked to articulate ideas relevant 
to the question(s) posed. This may be achieved without explicit prompting, or 
a framework like STEEPV may be used to stimulate ideas in different areas and 
to ensure no important area is overlooked. Aiming to provoke ideas in a wide 
range of areas might be appropriate for topics such as technology adoption 
and skills supply. On the question of skills demanded by types of work, it 
might be more effective to prompt ideas in skills areas, such as professional 
skills, manual skills, technical skills, teamworking skills and others. 
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In face-to-face brainstorm sessions, the usual approach is to ask 
participants to share their ideas verbally and record them on a whiteboard, 
flipchart, or computer display; this may be the task of a facilitator or member 
of the support team. Ideas may be reinforced or further developed in the 
process. 

A variant of this approach is to allow people some time to write their 
ideas on post-it notes or to do so using computer software. They may be 
able to consult ideas submitted by others when adding their own. After the 
time allocated has lapsed, participants may be given the chance to present 
their ideas to the group. It is good practice in both approaches to discourage 
critical reactions to ideas during brainstorming, so as not to inhibit creative 
reflection. Questions should be limited to requests for clarifications. 

Usually, the goal of the discussion following the collection of ideas is 
to decide whether to rule out some and how to group and prioritise the 
remaining ones. Ruling out some ideas does not mean abandoning them 
forever. Unless the owner of the idea concedes that it is inappropriate 
or irrelevant, it may still be reported. Ideas that are retained can often be 
clustered in groups, either because they are slightly varying formulations 
of the same basic point or complementary elements belonging to the same 
broader idea. 

When many ideas remain part of the discussion, establishing which ones 
are most important is a widely applied approach. For example, following 
a brainstorm on relevant policies or changes in curricula or training methods, 
participants may agree to focus on the feasibility of various ideas and their 
potential impact. In scenario analysis, it is common to use brainstorming 
to identify drivers of change with important but highly uncertain effects. In 
a skill forecasting exercise, drivers might include a reduction in prices and/ 
or increase in capabilities of technology, and the uptake of management 
approaches easing work organisation and organisational knowledge 
management. 

Figure 1 displays the results of an exercise aimed at helping UK health 
service managers think through issues that could impact their work and skills 
needs in the future at an early stage in their careers. The exercise followed 
a brainstorm aimed at identifying drivers of change (for example ageing 
populations and budget constraints). The bars indicate the extent to which 
participants believed different factors are important, and the colours give an 
impression about how certain they were about these factors materialising. 
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Figure 1. Factors influencing management work in health services in 
the future: views on topics identified in brainstorming 
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Source: Notes from workshop, conducted in 2010. 


44. The STEEPV framework 


The acronym STEEPV (social, technological, economic, environmental, 
political and values) refers to a convenient way of investigating and classifying 
phenomena and encouraging the generation of ideas that cover a wide range 
of areas (7°). 


(°°) The term STEEP is also frequently encountered as a reduced version of STEEPV. There is a family 
of such approaches, with other members known by acronyms, such as political, economic, social 
and technological (PEST), political, economic, social, technological, legal and environmental 
(PESTLE), or technological, economic, environmental, political, social and ethical (TEEPSE). 

Often a modified list of these terms is used to ensure adequate coverage of a particular topic. For 
further information regarding such foresight frameworks see Loveridge, 2002. 
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The aim of using STEEPV elements is not precision, but wide coverage. 
When asking participants in a workshop to brainstorm on drivers of change, 
the potential impacts of change, or the strengths, weaknesses, opportunities 
and threats (SWOT) associated with a particular policy, asking for ideas 
under all STEEPV headings can help the participants break away from 
a unidimensional focus or silo thinking. 

Grouping ideas about drivers of change under the STEEPV headings, for 
instance using posters arranged on a wall, helps visualise and structure the 
collective wisdom of a group. As a follow-up, participants can use stickers 
to indicate which change drivers they consider most important and (by using 
different colours) to provide an assessment of how likely it is different drivers 
of change will become dominant and impactful trends. Such an approach can 
greatly reduce the time needed for a group to agree on a set of critical issues. 

STEEPV has been widely applied in studies on technological change, 
employment and skills (Box 6). 


Box 6. Foresight studies using STEEPV framework 


A study of e-skills in Europe (CEPIS, 2006) identified many factors influencing supply 
and demand for such skills, and considered their implications, as follows: 

e social: improved education levels; expectations of higher wages; population age- 
ing trends; efforts to compromise systems/growing concerns about information 
and communications technology (ICT) security; 

technological: improved user interfaces; adoption of software engineering and more 
disciplined approaches; enhanced telecommunications infrastructure and services; 
economic: economic growth within the EU; increased start-up rate for small and 
medium-sized enterprises (SMEs); commoditisation of ICT products/services; in- 
creasing global competition; 

environmental: acceleration of global warming; pressure/effort to replace trans- 
port with telecoms; emphasis on reduction of ICT and electronic waste; 

political: variation of skilled worker migration flow(s) into EU; pressure exerted on 
employers to reduce/limit offshoring; growing geopolitical instability; extension of 
retirement age; 

values-related: declining interest of young people in undertaking technology-relat- 
ed courses/qualifications; demand from workforce for more satisfying jobs; resist- 
ance to surveillance/privacy invasion through ICT at work and in services; growing 
concern about health hazards of ICT. 


Source: CEPIS, 2006. 
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4.5. SWOT analysis 

SWOT (strengths, weaknesses, opportunities and threats) analysis is a useful 
approach to encourage workshop participants to break out of a mindset 
that overemphasises dangers or negative impacts of a development, or 
commercial prospects linked to it. SWOT involves asking people to examine 
systematically the arguments or possibilities associated with points of 
view different from those prevailing in their organisation or (professional) 
community. This helps identify issues that need to be taken seriously and 
counters any reflex to dismiss them without serious consideration. SWOT 
analysis is used to inform strategic planning and foresight activities, often 
those with a shorter-term focus. SWOT features can also readily be built into 
Delphi-type surveys or expert interviews. 

SWOT results are often presented in a 2x2 matrix, with strengths and 
weaknesses considered more internal and opportunities and threats more 
external factors. The strengths and opportunities cells are usually mainly 
positive, and weaknesses and threats refer to mainly negative aspects. 
Table 9 presents an example of a SWOT analysis of a vocational training 
scheme involving formal school-based and on-the-job factory-based learning. 


Table 9. An illustrative SWOT analysis of a VET scheme 


e Regular supply of highly skilled employees 
e Ensure sustainability of recruitment needs 
e Knowledge retention 

e Promoting new standards of education 

e Good experience in promoting VET 


Opportunities 


e Proximity to school 

e Possibility of obtaining public funds 

e Good example and opportunity with other 
schools 

e Cooperation with public organisations, e.g. 
Voluntary Labour Corps which is organising 
various forms of combatting unemployment 


Weaknesses 


e Lack of tutors due to retiring age 
e Lack of good dual education programme and 
system of education solutions 


e Students’ skills mismatch 

e Education programme mismatch 

e Low prestige of the apprenticeship education 
e Lack of training for teachers 

e Lack of well-prepared workshops in schools 


and social pathology 


Source: Vocational education and training — A renovated 


tradition in Poland. 


45 


46 


OO oO 


PREVIOUS CONTENTS NEXT 


Understanding technological change and skill needs 


Many tools for framing SWOT matrices, to help appraise possible 
futures and develop strategies, are available online. The SWOT matrix 
may be populated by an expert team, based on broader views obtained 
via a workshop. Sometimes analyses are predominantly based on desk 
research and data obtained via less participatory approaches, such as one- 
to-one stakeholder interviews. 

A common approach to processing the ideas present in the matrix is to 
prioritise strengths and weaknesses in terms of how influential each factor 
may be and opportunities and threats in terms of impact and probability. 
SWOT is widely regarded as a useful starting point in strategic planning, 
provided that those contributing ideas and perspectives are prepared to go 
beyond simply recycling conventional wisdom and/or repeating banalities, 
and willing to engage in fundamental work. Applying a STEEPV framework 
to stimulate thinking about possible factors may encourage reflection on 
a wider choice of issues. 


4.6. Collective creativity 


There are many techniques to encourage participants to be creative and 
contribute ideas about phenomena, events and actions that might not come 
to the fore in more routine discussions. Brainstorming is one of these but 
not all groups can use it effectively without preparation. While some groups 
spontaneously come up with quite original ideas in response to a request to 
think out of the box or imagine a really different outcome, in others this is 
not so straightforward and loosening-up activities are needed to encourage 
participants to feel free to construct new lines of thought. 

Creativity techniques range from asking participants to complete a simple 
task (for example ‘come up with as many uses for a brick as you can’), to 
more elaborate ones, such as role-play. One creative technique — the lotus 
blossom — is a framework of ever-widening ‘petals’ which are visualised 
using a set of flip charts or white boards. Workshop participants begin with 
a central theme, at the heart of the flower which is used to trigger new ideas 
and themes, which then become new central themes (Box 7). 

Lotus blossom, like many other creativity techniques, is a tool to explore 
various aspects of a problem or issue in an inclusive and systematic way 
and to go beyond discussions fixated on just one problematic aspect, issue, 
or stumbling block. In practice, limited time makes it challenging to examine 
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all facets or dimensions a creativity technique may identify. A pragmatic 
solution is to shape exercises so that they balance exploring new ideas and 
examining more familiar ones. 


Box 7. The lotus-blossom technique 


Figure 2 shows the set-up of a lotus-blossom flower exercise applied in a workshop 
on higher education. Its central aim was to develop ideas on how to create a truly flex- 
ible version of a training programme. Break-out groups worked on different goals. The 
first sub-themes to be developed on the basis of the central theme reflected different 
dimensions for achieving the main goal, for instance allow evening and weekend 
Classes, pay on a modular basis for training, relaxing constraints on course duration. 
Each group was encouraged to develop up to eight subthemes. For each of these 
new themes, participants were asked to suggest again up to eight actions. Sugges- 
tions included making scholarships more visible, addressing load transfer issues, and 
changing [specific aspects of] regulations and reward structures. 


Source: Unpublished 2015 study, conducted for UK university. 


Figure 2. Schematic representation of the lotus-blossom technique 
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NB: Sub-issues triggered by a central idea are captured in the circles A to H on the primary chart (represented by 
the matrix in the middle of the diagram), and these are transcribed onto the secondary charts (surrounding 
the central chart). 

Source: Cedefop. 
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5.14. Scenario workshops 


The term scenario is sometimes used loosely and appears to have different 
meanings in various contexts. The results of running a quantitative model 
several times with different assumptions or parameters can be labelled 
scenarios. Different points of view across different population groups, 
identified via survey research, may also be considered scenarios. In the 
context of scenario workshops, the term refers to more articulated accounts 
of future states of affairs and/or paths of development. Such workshops 
aim to develop internally coherent visions of possible future prospects. In 
foresight, the purpose of identifying and outlining scenarios is to capture 
the implications of plausible developments and courses of action. To reflect 
uncertainties about a topic, issue or trend, foresight scenario exercises 
generally consider a range of possible outcomes. 

A brief presentation or set of presentations on the topic of the exercise starts 
off most scenario workshops. This is followed by systematic discussion of issues, 
trends and drivers that are likely to influence future prospects; brainstorming 
and STEEPV are commonly used tools to structure such discussions. Apart 
from acquiring knowledge, deep immersion in key issues and adjacent topics 
of interest, this also gives participants the opportunity to learn more about other 
participants’ expertise and interests. In break-out groups, alternatives can be 
explored in some detail. Usually every break-out group considers one scenario: 
one course of development and its outcome. Some scenario approaches ask 
participants to assume that one or more trends are driving developments, and 
to reflect on how this might happen and what the consequences will be. Other 
approaches ask participants to imagine a particular state of affairs in the future. 
They are then asked to explain what processes may have led to this outcome 
and to reflect on how the future will likely unfold. 

As it is practically impossible to work through all possible scenarios, the 
challenge is to find exemplary ones. It is important to ensure that these are 
not only seen as plausible prospects, but as states in a wide spectrum of 
possibilities. The dimensions in this spectrum may not be the ones that turn 
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out to be most significant. Most participatory scenario exercises work with 
three or four alternative scenarios. 

In foresight studies, two types of scenario workshop are widely used. 
Many exercises use a 2x2 driver approach, which leads to four scenarios. 
This requires identifying two drivers which are seen as particularly important 
for the topic and, at the same time, highly uncertain in terms of development 
and/or impact. For each of the two drivers, polar directions of development 
are identified; each of the four scenarios represents one combination of the 
two sets of directions. 

While the second type of scenario workshop may also be preceded by 
a discussion of major drivers, the scenarios are not built based on alternative 
driver development paths. Participants are instead provided with a set of very 
basic alternative outcomes and asked to envisage the drivers and events 
that could have led to these outcomes. Subsequently, they reflect on what 
the outcomes would possibly look like in more detail. The outcomes could 
be based on alternative end-points established in earlier work, for example 
centralised/decentralised, public/private, technology/social-innovation based. 

As part of a 2006 study, several scenarios on supply and demand 
for e-skills across Europe, developed through deskwork and supported 
by computer modelling, were presented at a workshop (CEPIS, 2006). 
Participants were asked about factors influencing the alternative paths 
of development. One generic approach that proves useful in many 
circumstances is to set up three (or four) archetypes: future states of 
affairs that are better than expected, worse than expected, different than 
expected (and/or radically different than expected). It is not uncommon for 
the different options to result in a major shift in participants’ understanding 
of the prospects for evolution (2'). 

All scenario approaches have the common aim to generate scenarios 
that participants consider plausible. Rapporteurs from breakout sessions 
will typically present the group’s scenario to the plenary, trying to ‘sell’ it as 
being something that workshop participants and decision-makers should 
take seriously into account. 


(¢") For example, in one exercise where the prospects for a particular type of network technology 
were being examined, the participants had been focusing on the extent to which the strategies 
that could be adopted by the major existing players would succeed in their intended aims of 
reaching large user bases and encouraging new applications. The result of the scenario exercise 
led them to consider seriously the possibility that a disruptive newcomer could provide an 
alternative set of facilities that would completely change the situation. 
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The process aims to capture as many alternative scenarios as possible, 
reflecting on which drivers operate, how, and with what outcomes. The 
framework used should facilitate Comparisons between alternatives, 
possibly using tables, graphs or charts. Such comparisons can be useful 
in communicating uncertainties and avoiding a focus on business as usual 
or most likely expectations. Alternative scenarios can be used to examine 
the robustness of policies. Often a scenario workshop will culminate in 
participants being requested to suggest early warning signs and actions that 
might be necessary should one or another scenario emerge. 

The development of action recommendations typically focuses on the 
organisation/group benefiting from the exercise, and/or a wider stakeholder 
group needing to prepare for and respond to alternative futures. The 
descriptions of the scenarios, and the factors leading to their development, 
can be used effectively to convey results of the foresight exercise to a wide 
audience. They may be illustrated with vignettes (accounts of events or 
experiences in the future), cartoons, or other visual aids. 

It is also possible for scenario work to be closely linked to personal 
choices of participants. Those with a particular occupation or engaged in 
a certain sector may be asked to reflect on what different scenarios might 
mean for their own career and the skills and training they will need. 

In scenario workshops, wild cards collected during discussions could be 
retained for future reference and can be useful for risk appraisal exercises. 
Representing events unlikely to occur, wild cards are not usually pivotal in 
constructing plausible scenarios: the future is likely to look like a mixture of 
different scenarios. Scenario workshops should help clarify what underlies 
these possibilities and what factors may shape their future development. 
While many practitioners will ask participants to estimate the likelinood of 
alternative scenarios, it may be more appropriate to ask what the future 
may look like according to one or another scenario or resort to voting. While 
elements from some scenarios tend to be more prevalent, all will have some 
features that are thought to be indicative of the shape of things to come. 

Scenarios produced from desk research and other non-participatory 
methods, or in earlier scenario workshops, may be presented at workshops 
to be validated or elaborated. This may entail reflecting in more detail on the 
implications of trends for specific occupations, industries and related areas 
(such as training systems). Often a workshop convened to develop strategic 
recommendations can build on prior scenario work and a desirable scenario 
or roadmap can be constructed from it. 
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5.2. Simulation gaming and role-playing 


Computer simulations are a widely used tool for modelling future 
developments. Most are developed by expert teams; in some, participatory 
elements are present in building them. Since the 1970s, teams of experts 
developed such models. These were presented to national policy bodies and 
experimenting was encouraged to use the models to see the impact on their 
policy decisions. 

Today’s commercial and educational computer games are based on 
sophisticated models and allow users to visualise developments. Some 
teach players how economies, for instance, operate. Large-scale multiplayer 
online games allow large numbers of people (present in workshops and 
remotely) to participate in simulations and to build (virtual) future worlds. In 
fields like urban planning, using such games can be a real value-added. 

In practice, simulation gaming in workshops often encourages 
participants (or groups of participants) to act out certain roles. In the context 
of technology and skills foresight, this may involve imagining being users 
of a specific technology and considering how to deploy it. The role-playing 
tools and approaches commonly used in psychotherapy, counselling, and 
related areas can be adapted for use in a skills context. Participants could, 
for example, be asked to imagine: 

(a) what it would be like to be a person of a particular age and occupation, 
with a particular life history, confronted with a particular technology; 

(b) what the challenges are in using that technology; 

(c) what skills are required; 

(d) what may incentivise the choice of investing in different skills (2). 


Gaming in futures studies developed out of war-gaming exercises in 
military contexts, where players take opposing sides in a conflict situation 
and explore consequences of choices and reactions to these choices. To 
model social phenomena, simulation exercise can assign participants: 

(a) the roles corresponding to those of social groups in situations of interest; 
(b) the motives and information available to these roles; 
(c) the options for action they have. 


(??) The persona approach is commonly applied in marketing studies, service design and open 
policy-making. 
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With participants performing the roles of key agents within a structured 
framework, possible responses to emerging circumstances can be explored. 
Interactions between the strategies of different agents can be examined. 
So far, there is limited experience in using these methods in a foresight 
context. It may be that their main functions are educating participants about 
contingencies and motives, so that they develop insight that can feed into 
other foresight activities and outputs. 


5.3. Actions, decisions and recommendations 


5.3.1. Suggesting and developing action plans 

Suggestions for possible actions emerging from a foresight analysis are 
often made in the concluding stages of workshops. Developing actions 
and action plans may also be undertaken in dedicated workshops following 
earlier foresight work. 

Ideas may be developed using the methods described, such as 
brainstorming and the lotus-blossom approach. These techniques encourage 
development of creative ideas for action, while systematically considering 
the complexity of many objectives. 

Action plans are lists linking actions to stakeholders that have (or could 
have) capacity and responsibility for executing them. The carousel method is 
sometimes used: this involves workshop participant groups rotating between 
flip charts or white boards representing different stakeholders and actors. In 
the process, they inspect the contributions of other groups and add their 
own suggestions for actions that stakeholders or actors might undertake 
to realise a particular goal (or simply a more desirable future). This allows 
different groups to express their own viewpoints and perspectives, as well 
as giving participants the opportunity to use their legs, which is often much 
welcomed after being seated for many hours. 

Effective action plans are not just wish lists of desired end points. They 
are specific and, alongside the detail, explain how intended outcomes can 
be achieved. Providing measures or indications of how to monitor progress 
(or problems) and targets to be achieved is beneficial. Some practitioners 
recommend setting ‘stretch’ targets, which have to be ambitious but 
attainable, provided sufficient effort is made. It is important to avoid unrealistic 
actions and goals, particularly in a context where sudden impressive 
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performance is unlikely. Examples are lack of resources, insufficient political 
will to accomplish something, or a history of institutional failure. 

Sound knowledge about institutional capabilities and interests is 
important. It enables formulating actions that are reasonable and attainable, 
which can be presented to decision-makers. It is helpful to include 
participants who are familiar with the organisational culture, language, 
procedures and timetables of key institutions. Proposals for action must 
respect current norms and practices, avoid treading on the feet of key 
stakeholders and actors, and should not replicate policies and programmes 
which have been shown not to be very effective. Knowing how actions 
might fit into political and policy cycles is valuable. Proposing several new 
activities to an organisation which has just revised its strategic planning may 
be of limited value. In other situations, suggestions for change or innovation 
may be exactly what is required. 


5.3.2. Open policy-making and policy labs 

The open policy-making approach was developed in the UK civil service 

to encourage more rapid and creative development of policies, including 

those related to changing skill needs in an increasingly digital world. As in 
participatory foresight, open policy-making aims to meet user needs through 
collaborative approaches that inform policy thinking based on a broad 
range of expert inputs. These complement data-driven and evidence- 
based approaches relying on new analytical techniques and digital tools. 

Prototyping and iteratively improving policies are used to meet complex and 

changing user needs (°°). 

One of the methods employed is the Policy laboratory in a day. This 
method involves: 

(a) setting the challenge: participants use post-its or similar tools to express 
hopes and fears and then identify the challenge that is being confronted. 
Voting is used to identify the key challenge that will be the focus of the 
discussion; 

(b) understanding the policy landscape: break-out groups capture key issues 
about data and evidence (what is available, what is most important/ 
interesting/surprising, where are there evidence gaps?), users and 
stakeholders (who is targeted, which other key stakeholders are there?) 


(5) For a toolkit of such methods, and links to other resources, see: https://www.gov.uk/guidance/ 
open-policy-making-toolkit 
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and existing policies aimed at addressing the challenge. These are 
discussed to map comprehensively the current policy landscape and to 
identify possible desired interventions; 

(c) understanding the users: break-out groups develop personas (types of 
actor influenced by the policy challenge), their ‘journeys’ (experiences 
and needs related to the policy areas), and how these might influence the 
policy challenge; 

(d) generating ideas: break-out groups brainstorm about major ideas for 
policy actions, prompted by change cards. These are clustered and the 
top three are selected. Ideas are visualised through drawing, for instance 
in the form of a service journey depicted as a cartoon and pitched to the 
whole group; 

(e) planning next steps: groups explain immediate next steps (in terms of 
hours and days rather than weeks) and what is needed to achieve them; 
for example, the people who will need to engage, further work needed 
to understand better the policy challenge and the users. Timing and key 
milestones help map an action plan with timetable. 


Box 8. The UK foresight project on skills and lifetime learning 


The UK foresight project on skills and lifetime learning used a version of the policy- 
laboratory-in-a-day approach in its Lifetime learning in the digital age summit. 
Accepting that changing technologies raise the uncertainty of future digital skills 
requirements, participants expressed hope that such skills will be acquired in more 
inclusive and accessible ways in the future. Three broad topics were considered: 
the incoming flow of skills from workforce entrants; skills for those already in the 
workforce; and demand for skills from employers. After review, actions were pro- 
posed for each. 


In terms of incoming skills flow, actions were identified complementary to current 

policies. 

e Teachers should be offered training to address their skills gap. 

e Industry-provider partnerships, where firms articulate their digital skill needs and 
providers respond to them. 

e Targeting underrepresented groups for computing degrees, including women, eth- 
nic-minority groups, disabled people and individuals from neighbourhoods with 
low participation rates, by ensuring the education system provides opportunities 
to improve diversity in uptake of subjects, such as maths. 
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e Improving the digital competence of higher education students in all subjects, for 
example by including digital content in degree programmes. 

e Building work-readiness into computer science or science, technology, engineer- 
ing, mathematics (STEM) degree programmes. 

e Increase sandwich courses and work experience options. 


For those already in the workforce. 

e Encouraging and supporting employers to provide better workplace learning. 

e Codification/accreditation, for example digital badging, so that individuals, em- 
ployers and education providers can transfer learning from outside the formal 
system and between workplaces. 

e Developing self-identification skills so that individuals are more able to diagnose 
which skills they need, lack and can acquire. 

e Supporting self-learning of technology through community groups. 

e Supporting learning at or around significant life stage transitions which influ- 
ence participation, and can contribute to learning motivation (including childbirth, 
changing job, redundancy, children leaving home, bereavement, retirement). 

e Supporting intergenerational and family learning. Older adults are often motivated 
to learn to support their children. 

e Improving the provision of part-time digital courses at higher levels (further or 
higher education) where the recent trend is a reduction in the number of part-time 
programmes. 


Demand for digital skills from employers. 

e Improving information on which digital skills and capabilities are available for em- 
ployers, including how they could help their organisations become more produc- 
tive and grow. 

e Emphasising the importance of technology ‘translators’ in organisations: staff 
members who are able to bridge the gap between understanding new technolo- 
gies and the possible impact they may have on value generation. 

e Intermediaries can share and promote understanding of digital skills requirements 
between employers, providers and employees. 

e Atthe regional level, local government and local enterprise partnerships can develop 
amore detailed understanding of digital skill requirements. There could be opportu- 
nities to link universities, colleges and other education providers to local employers, 
and design training programmes geared to the needs of the local economy. 


Specific policies to support skills development for the digital and creative industries. 
e Further development of specialist technology institutions and applied research, 
learning from institutions like MIT, Pohang (in South Korea) and ETH Zurich. 
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e Attracting high-skilled international talent to specific sectors. 

e Improve the links between digital SMEs and education and training providers, for 
example assisting graduate integration into local SMEs via placements and on- 
the-job training programmes. 

e Developing creative digital skills (not just digital skills) in line with research sug- 
gesting digital employers increasingly want individuals with a mix of technology 
and creative skills. 


In summary, the open policy lab concluded that ‘interventions will be required across 
the digital skills landscape’ (Government Office for Science, 2017, p. 12). If such 
interventions are to achieve the needed improvements in the skill framework, there 
will need to be much collaboration across the various organisations involved on both 
sides: supplying skills and employing them. This key finding fed into the more gen- 
eral analyses of the foresight project. 


Source: Government Office for Science. 2017 


5.3.3. Roadmapping 

As with many other terms in foresight, roadmap can have various meanings 
in different contexts. In some projects, the general framework, developed 
based on discussions on pursuing a set of policy goals, is called a roadmap 
(Box 9). Most foresight outcomes referred to as roadmaps do not 
systematically deploy the methodology of this foresight technique. 


Box 9. The European roadmap 


The European roadmap, produced by the Straighten basic skills (SBS) project, aims 
at increasing ‘participation in the public consultation on questions related to the 
strengthening of work-related basic skills, and the training participation of low-qual- 
ified employees and unemployed in adult education’ (SBS, 2017, p. 1). It documents 
the implementation of basic skills education in European countries and outlines ‘key 
factors of social, economic and cultural elements which contribute to the success 
of work-related basic skills provision’ (SBS, 2017, p. 1). These are framed in terms 
of success indicators, generated by desk research and discussions in a practitioner 
workshop. The European roadmap is a general framework, which can be used to 
shape more detailed national roadmaps. 


Source: SBS, 2017. 
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Roadmapping is best known though its application in developing 
technology or policy roadmaps. It begins with a core objective, typically 
a statement that a given state of affairs will be achieved by a certain date. 
Roadmapping looks into how actions could be sequenced. Different layers, 
referring to different types of action or actors, are used to structure the 
analysis. For instance, in the case of technology roadmapping, layers can 
refer to research and development, the regulatory framework, consumer 
markets or user applications and skills requirements. 

The outcomes of different actions (for example development of 
technological capabilities, establishment of regulatory or other standards, 
take-off of markets) are related and some are dependent on others. Aroadmap 
diagram visualises these actions and outcomes as they evolve over time 
across layers. A roadmap may reveal that objectives are insufficiently or 
overly ambitious, in terms of how progress or impact is to be achieved at 
a given point in the future. 

In practice, a range of approaches can be used to develop roadmaps. 
Some are produced by small expert groups. Others take a wider approach 
or are informed by the results of Delphi surveys, which can help in assessing 
when various developments are expected to occur and when they are likely 
to become mainstreamed. Participatory approaches, such as workshops, 
are particularly useful because they bring together expertise and knowledge 
relevant to different action layers. 

Several tools, including computer-aided and pen-and-paper ones, 
are available for roadmapping exercises, to support understanding of the 
interdependencies in complex systems. As a fully fledged roadmapping 
exercise is laborious and time-demanding, in practice many roadmaps are 
developed using abridged versions of the approach. 

Using the success scenario technique, which involves setting a stretch 
target for the topic at hand, is a pragmatic solution. This technique considers 
the actions needed to achieve the target and the success indicators implied 
in different fields or across different (groups of) stakeholders (74). 

Employment and skills can be incorporated into technology roadmaps, 
with possible layers reflecting training provision and uptake and skills 
utilisation in different roles and phases of the technology cycle. Most 
technology or industry roadmaps tend to be produced independently at first, 


(4) Such an approach has been used, for example, in a workshop used to develop a strategic 
appraisal of the UK’s potential in nanotechnology. See Advisory Group on Nanotechnology, 2002. 
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with a subsequent focus on the implications of technological drivers for skill 
supply and demand (?°). 


5.3.4. Choosing among actions 

At the end of a workshop, there may be many great ideas or relatively few. 
These may include creative proposals for actions to be undertaken, risks to 
be aware of, topics to be further investigated or issues to reflect in education 
and training curricula. It is unlikely that all ideas have similar importance. 
The target audience is also likely to have limited attention and absorption 
capacity. While it is desirable to present a wide range of creative ideas, many 
decision-makers will not react positively to a long list of possible options (7°). 

A key problem with using simple voting or polling (See also Chapter 4) is 
that what is put on the ballot may vary in terms of quality. A broader formulation 
will often gather more support than a narrower one. Before engaging in 
a voting exercise, it is crucial to review formulations carefully, to eliminate 
redundancies and repetitions, reduce ambiguities, and, as far as possible, 
achieve similar levels of abstraction across topics, issues and trends. 

It can be challenging to define the categories to be used to indicate 
choices. One common approach is to contrast ideas in terms of few 
summary criteria (drawing upon several sub-criteria). In practice, workshop 
participants are often asked to cast judgement using two or three categories 
(for example yes/no, small/moderate/large, or negative/neutral/positive). 
More detailed judgements may be collected by using rating scales; this 
tends to be complicated in workshop settings, unless IT-based solutions 
can be used. 

Criteria typically used in collecting information aimed at understanding 
how stakeholders view particular issues or ideas include: 

(a) impact on one or more objectives or issues of concern; 
(b) feasibility of, or costs associated with, activities; 
(c) originality or novelty of ideas. 

Figure 3 illustrates how such criteria can be used in practice. In a foresight 

exercise on challenges in higher education, participants were requested 


(5) An example includes the work around transition to more sustainable and low-carbon energy 
production and use in Europe (Energy roadmap 2050) and a number of major European 
countries. 

(76) It is not unknown for senior decision-makers who are wrapping up workshops to request that 
participants tell them what they consider to be the one outstanding action point to emerge from 
the workshop. 
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to place options in the appropriate cell of the 2x2 matrix based on two 

criteria. The vertical dimension was for ease of implementation of the idea 

(lower=easy, upper=difficult), and the horizontal dimension for originality 

(left=quite normal, right=original). This produced four categories: 

(a) Category 1 — ideas to be discarded as both unoriginal and unfeasible; 

(b) Category 2 — ideas for the future, dreams, challenges, stimulation for the 
brain; 

(c) Category 3 - ideas that are easy to implement, low risk, high acceptability, 
done before; 

(d) Category 4 - ideas that are innovative, breakthrough, exciting, can be 
implemented (?”). 


Figure 3. Choice matrix 


CATEGORY 3 CATEGORY 4 
CATEGORY 1 CATEGORY 2 


Horizontal dimension 


Source: Cedefop. 


(’) Inspired by 27 creativity and innovation tools in one-pagers, p. 24. 
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Box 10. Higher education example using a choice matrix 


In a foresight exercise dealing with new challenges and new courses in a higher 
education context, some of the activities classified in different categories included: 
Category 2: training of ambassadors among graduates; strategic development of 
partnerships and pilot programmes; and investment in facilities that adequately re- 
flect the determination and quality of the organisation. 

Category 3: examining experience in overseas institutions facing this challenge; pro- 
vision and targeting of scholarships; and reduction of internal barriers to multidisci- 
plinary work. 

Category 4: bringing in high-profile individuals to give lectures and lead events; en- 
hancing student support to increase retention; organising institution-wide foresight 
events addressing the topics, ensuring participation of senior leadership. 


Source: Unpublished 2015 study, conducted for a UK university. 


Another way to use such a matrix in policy and business contexts is for 
the horizontal dimension to represent impact (usually in terms of economic 
benefits, but it is also possible to use ‘share of the workforce affected’ as 
impact criterion) and the vertical dimension feasibility or cost of the activity. 

In this case the four categories can be characterised as follows: 

(a) Category 1 - ideas here are less attractive, with low potential to achieve 
them, and low impacts. However, these may be areas to monitor: for 
instance, technologies might improve and applications proliferate, 
they might become easier to use or more worthy of skill development 
investments; 

(b) Category 2 - ideas here are of high interest and may be regarded as 
‘low-hanging fruit’, where obvious benefits may result relatively rapidly. 
Implementation could begin immediately; 

(c) Category 3 - ideas here are fairly low-risk and could have benefits in 
specific areas, even if overall impacts may appear limited; 

(d) Category 4 - these are areas where most effort may be needed to achieve 
wide and long-term impacts; they will typically require considerable 
investment and planning. 


Such a matrix approximates the prioritisation criteria typically employed 
in critical technologies approaches, and it is possible to build assessments 
of different dimensions of feasibility/cost and impact/benefits into a multiple 
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criteria analysis framework (Section 5.3.5). Examples of such application to 
various topics, including those concerned with training, education and skills, 
are numerous and useful for inspiring new foresight initiatives (7°). 


5.3.5. Multiple criteria analysis 

Multiple criteria analysis (MCA) is a more elaborate way of considering 
alternative ideas. It looks at fewer ideas but in much more detail and using 
more criteria to inform decision-making. Like many other judgemental 
techniques, MCA can be used with individual stakeholders or be part of 
Delphi or other survey studies. It can also be embedded in workshops, 
particularly where there are not too many issues or criteria to be employed, 
and when groups can share the work. The face-to-face discussions in 
workshops ease insight into what underpins judgements, which can help 
clarify what is being appraised or assess the evidence on which opinions are 
based. The technique is simple and involves assessing a set of actions or 
development paths in terms of set criteria. 

The criteria are often outcomes of interest, such as creation/reduction of 
employment, in particular occupations or industries, increased/decreased 
demand for particular skills or any other policy- or business-relevant topic. 
Typically, the options are assessed in using rating scales for the criteria. The 
different criteria may be weighted, so that overall scores can be obtained, 
and particular options prioritised based on, for instance, economic or 
environmental or social welfare benefits/costs. 

Apart from the simpler approach of simply averaging responses collected 
in workshops, it is also possible to ask groups to decide how to reflect different 
outcomes and the weights of different criteria. Findings based on responses 
of larger expert groups, obtained via surveys and interviews, may be more 
convincing for audiences than those based on few responses gathered 
in small group discussions. This does not mean such discussions are not 
valuable in a broader sense. Small group discussion, using the systematic 
approach MCA offers, stimulates profound learning and reflection. 


(-®) Examples at: 
(a) https://hbr.org/201 7/09/a-2x2-matrix-to-help-you-prioritize-the-skills-to-learn-right-now 
(b) https://www.bu.edu/tech/about/service/incident-management/managing-tickets/ 
priority-matrix/ 
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Box 11. Multiple criteria analysis in the FISTERA project 


MCA was used to build a priority matrix in a scenario workshop undertaken in the 
context of the FISTERA project. Participants assessed feasibility in terms of what 
was dubbed the PREST framework (political feasibility, resources (economic fea- 
sibility), ethical constraints and values, sustainability and technological feasibility). 
The impacts examined were sustainability/environmental quality, social cohesion, 
job creation, economic growth/wealth creation; competitiveness and innovativeness, 
employer-employee relations and work-life balance (2°). Small break-out groups pro- 
vided their ratings using an IT platform. 

Among actions rated high in terms of feasibility and positive outcomes were (Green 

et al., 2005): 

e EU to sponsor education of end users in security policies for user empowerment 
(to control the process) at EU level to balance security and privacy, and to educate 
users about regulations, and to generate competence in the communities’ public 
debate, awareness raising, ethical curriculum to educate young people. 

e Training for information society technologies at all levels in society, including SME 
development, with educational programmes at all levels including universities. 
This includes lifelong learning for pleasure and for the workforce. 


These were among the least well-formulated proposals. Most others (for instance, 
use of information technologies in the public sector and the push for various stand- 
ards) were more concisely formulated. These examples illustrate a limitation of pri- 
oritisation approaches. Although all proposals were the product of lively discussions 
and concrete ideas, activities proposed may well vary considerably in terms of their 
breadth. Broad activity may attract a great deal of support, while something more 
specific attracts little. While feasibility assessments should moderate this effect, it 
may be difficult to assess feasibility of broad formulations because they are impre- 
cise. Therefore, it is good practice to remain critical and aware of methodological 
limitations when presented with a priority list. 


Source: Green et al., 2005. 


(9) The same goals had been used to contrast different scenarios and Lisbon agenda goals (job 


creation, wealth creation, competitiveness, social cohesion, environmental quality, and social 
inclusion) in a Delphi survey concerning impacts of various applications of ICT (Popper and 
Miles, 2005). 
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5.4. Putting foresight findings in practice 


5.4.1. Consensus building 

After crafting an action plan or policy roadmap, a final and crucial stage 
of the foresight process is reaching consensus among key stakeholders on 
the best way to proceed with implementation. The success of consensus 
building critically depends on a range of factors and conditions (Table 10). 


Table 10. Success factors in consensus building exercises 


Establishing It is vital to begin the process by stating what the goal is for each 
a common goal participant and try jointly to define the common goal. 

Reaching consensus is a time- and energy-consuming process and 
Commitment to requires commitment. Participants must be honest about their positions 


reaching consensus _and actively listen to the opinions of others. They should be open to 
slightly shifting their positions if needed and expect others to do the same. 


Participants need to trust each other and know they are working towards 
a common goal. No participant should try to manipulate the process for 
own gains. This requires openness so that all participants have an honest 
and holistic understanding of their respective wants and needs. 


Participants should allocate enough time for taking part. This avoids 
having to revisit decisions made in earlier stages later in the process. 


Clarity about process _ It is important that all participants understand the process and the 
and methodology methodology used. 


All participants should listen to others, participate actively in the 
discussion and voice their concerns and thoughts whenever relevant. 


Where there is a large group, a facilitator should be appointed to guide 
the process and make sure its goals are met. 


Trust and openness 


Sufficient time 


Active participation 


Good facilitation 


Source: Cedefop 


In small groups (up to 15-20 people), consensus can be achieved with all 
members coming together and discussing different opinions and viewpoints. 
When people cannot meet physically, or cannot complete the discussion 
because of time restrictions, a Delphi-type consensus building method 
incorporating feedback loops in several sequential rounds may be used (°°) 
(Section 3.2.2). The typical steps involved are illustrated in Figure 4. 


(°°) Hsu and Sandford (2007) generally suggest that a minimum of 45 days will be needed. 
Moreover, they recommend giving subjects around two weeks to answer the questionnaires. 
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5.4.2. Communicating foresight findings with impact 

Communicating foresight findings effectively goes far beyond simply 
presenting information. It also involves awareness raising among 
stakeholders and wider audiences: participants should transmit the main 
messages of the activities to their own constituencies and organisations. 
Efficient dissemination of results can take the form of releasing publications 
via conventional publication outlets or online media. 

To promote or disseminate findings further, foresight participants can be 
encouraged to take part in, or even organise, public meetings and/or more 
specialised workshops and conferences. Foresight may also engage a wider 
set of intermediaries, such as trade unions, industry associations, education 
specialists and opinion shapers to help promote key messages. Although 
such wider outreach may happen spontaneously later during a foresight 
exercise, it is better to plan it at an early stage. 

Quality of presentation, lucidity, accuracy of accounts and conciseness 
of texts drive the effectiveness of foresight documentation. Knowledge of 
target audiences is also critical. Offering opportunities for feedback, valuing 
it and using it whenever possible builds trust and contributes to creating 
a positive image of the exercise. Cooperating with professional publication/ 
marketing professionals is beneficial. 

Packaging several actions into a coherent set - a demonstrator — is 
one way of avoiding a lengthy list of actions and helps organisations and 
decision-makers move forward. Demonstrators also support evaluation of the 
proposed actions and can be a basis to pilot or stepwise implement them. 

From a wider perspective, evaluating foresight exercises is important for 
understanding organisational or economic impacts. It is good practice to 
draw on participants and end users’ feedback, especially in terms of what 
they have learned from the process, and how it contributed to establishing 
new networks and relationships. The insights on altered perceptions and 
practices they offer, and their continued support after foresight has been 
completed (for example monitoring the take-up or implementation of actions 
plans), are valuable. An important achievement of every well-conducted 
foresight exercise is its contribution to fostering a foresight culture. A context 
where communities and decision-makers are aware of the benefits of 
foresight will help ensure it is used whenever economies or societies can 
benefit from the insight it offers. 
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Figure 4. Consensus building process 


Step 1: Introduce, clarify and agree on the issue(s) to be decided and the rules of the 
process 


Make sure that all participants agree and are aware of what the main discussion is about and 
what are the key questions 


Step 2: Explore the issues and look for ideas 


e Gather initial thoughts and reactions regarding people’s concerns about the issues they agreed 
to discuss 

e Collect ideas for solving the problem and put them in writing 

e Have a broad ranging discussion on the topics and debate the different ideas. What are the 
pros and cons? 


Step 3: Look for emerging proposals 


Look for a proposal that weaves together the best elements of the ideas discussed. Look for 
a solution that addresses people’s key concerns 


Step 4: Discuss, clarify and amend the proposal 


Ensure that any remaining concerns are heard and that everyone has a chance to contribute. 
Look for adaptations that make the proposal even more acceptable to the group 


Step 5: Test for agreement 


Did the group manage to reach an agreement? Check for the following: 

Blocks: these imply fundamental disagreement with the core of the proposal which has not been 

solved previously and therefore a new proposal needs to be found 

Stand asides: these persons cannot support the proposal but they also do not want to stop the 
No | group and will therefore let the decision proceed without them 

Reservations: people might have certain reservations but will nevertheless let the proposal pass 

Agreement: shows support for the proposal and willingness to implement it 

Consensus: it can be said that consensus was reached when there are no blocks, not too many 

stand asides or reservations and active agreement 


Was Consensus reached? 
Yes 


Step 6: Implement the decision 


Who, when, how? Decide on an action plan and set deadlines and milestones 
This step can also commence in steps 3-5 and therefore be part of the overall process 
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This third Cedefop guide on methods for identifying technological change 
and its impact on skill requirements has looked at the value of technology 
and skills foresight approaches. Four important points need to be borne 
in mind when considering foresight analysis in the wider context of skills 
anticipation methods. 

First, while skills foresight integrates purely quantitative approaches to 
skills anticipation, it is fundamentally different from them. A long-term labour 
market forecast can be conducted by an expert group with little recourse 
to participatory methods. In contrast, it is rare to engage in participatory 
foresight without some use of information stemming from non-participatory 
skills anticipation methods. Anyone engaging in future analysis will implicitly 
draw upon earlier studies, materials and discussions. 

Second, a wide spectrum of tools and techniques can be used in either 
participatory or non-participatory settings. At one extreme are exercises 
with low levels of participatory engagement, such as those relying on 
a small stakeholder or expert group. At the other extreme, skills anticipation 
approaches involve large numbers of people representing a wide range of 
stakeholders. 

Third, foresight is a process, and different steps in the process may be 
organised in more or less participatory ways. In the entire process, and at 
the level of the steps it includes, the purpose determines which particular 
techniques can be successfully applied. Sometimes a foresight exercise 
includes components that are best shaped using non-participatory methods; 
it is important to share these with foresight participants. 

It is important to be aware of the fact that many assumptions that are 
routinely made in standard non-participatory quantitative approaches are not 
appropriate where it comes to dealing with technological change and social 
innovation. Such phenomena disrupt (Supposed) equilibria, introduce new 
variables and are usually only reflected by crude proxies in standard analyses. 
Reliance on just one type of expertise runs the risk of unjustly reflecting the 
narrow assumptions of that expertise into the entire process. This is a key 
argument for encouraging broader participation and multidisciplinary work. 
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Fourth, the question of who participates in foresight is a central one. One 
major reason for relying on participatory methods is to access knowledge 
from a broad range of expertise and to shed light on the motivations and 
capabilities of various stakeholders. These experts and stakeholders may 
be important actors for disseminating the results of foresight exercises. In 
many cases, they can also help implement the recommendations in their 
own organisations or communities of practice. 

When considering foresight analysis, the checklist presented in Table 11 
can be used. 

Conventional skills forecasts and surveys (see first Cedefop guide, 
2021a) and big data and artificial intelligence analysis (see second Cedefop 
guide, 2021b) are a rich source of information on technological changes and 
skill demands in labour markets. Information developed, based on these 
skills anticipation techniques, complements technology and skills foresight 
methods, which tend to address policy-relevant questions with a longer- 
term horizon. By engaging with key stakeholders, the process of technology 
foresight offers opportunities to reflect critically on findings based on non- 
participatory approaches. There is potential to arrive at novel insights about 
how the future will unfold and how policies and actions need to develop to 
build on positive features, while addressing unwanted negative effects. Such 
methods give those with a responsibility for skills governance or intelligence 
the tools that help them match future skills supply and demand in ways that 
benefit society. 

Qualitative participatory and quantitative non-participatory methods are 
not mutually exclusive. Ideally, they should support one another so that they 
can potentially shape an iterative process whereby stakeholder engagement 
can ease data collection and analyses in the non-participatory stages (and 
vice versa). Such interaction makes it possible to develop views on how the 
future will unfold and in what direction skills policies and actions need to 
develop. In many respects the challenge is to make effective use of the wide 
variety of data and information available. 

Table 12 provides a summary to guide policy-makers and analysts in 
understanding when to use the approaches covered by Cedefop’s skills 
anticipation practical guides. To learn more about conventional labour 
market and skills anticipation approaches (skills surveys, skills forecasting) 
and big data and Al-based methods, readers are referred to the other two 
Cedefop ‘how-to’ guides. 
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Table 11. Issues to consider before undertaking technology/skills 
foresight analysis: a checklist 


[Checklist = Issues and questions to be considered 


¢ Determine why a participatory foresight type approach is required. Is 
Initial check there a lack of consensus on the impact of technologies on skills, or 
a high unknown factor that needs to bring together various groups? 


Types of question 
that can be e What dominant technologies are likely to emerge in the medium term, 
addressed regarding either in the economy or in a particular sector or locality? 
technologies 
e What are the skill needs that are likely to be linked to the various 
7 technologies of interest? 
Ae Bi ee e Does this involve skills transversally required in many jobs/ 
occupations/sectors or skills required in niches which are critically 
addressed about ‘mnartant? 
skills Ih sean 


e What skills may no longer be needed to the same extent in the future? 
e What needs to be in place for skills supply to keep up with demand? 


e Background information that can be used to inform the foresight 
Requirements process 
e Willingness of groups to work together to find solutions 


e Deciding on participation: who should be involved and why 

e Initiation: scoping exercise, determining main foci, intended uses and 
users 

e Intelligence: literature review, horizon scanning, trend analysis 

e Imagination: establishing understanding of linkages between issues 
and using creativity to explore implications 

e Integration: development and appraisal of possible futures 

Interpretation: identification of strategies and prioritisation of actions 

e Intervention: discussion of proposed strategies, priorities, and next steps 
with key actors. Establishing roadmaps and implementation plans 

e Impact: evaluation of impact of foresight, identification of follow-up 
activities 

e ETF; Cedefop; ILO. Developing skills foresights, scenarios and 
forecasts. Guide to anticipating and matching skills and jobs: volume 2. 
Luxembourg: Publications Office. 

© Miles, |. (2010), The development of technology foresight: a review. 

Selected further Technological Forecasting and Social Change, Vol. 77, No 9, 

information pp. 1448-56. 

e Miles, |., Keenan, M.; Kaivo-Oja, J. (2003). Handbook of knowledge 
society foresight. Dublin: European Foundation. 
© Miles, |.; Saritas, 0.; Sokolov, A. (2017). Foresight for science, 
technology and innovation. Berlin: Springer. 


Foresight steps 


Source: Cedefop. 
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Table 12. A menu of skills assessment and anticipation choices 


Type of Capacity to hc 


Quantitative, non-participatory approaches 


When there is a relatively 
well-developed 
understanding of the 


Tend to be good at collecting 
information about recent past 
and impending changes. Not 


Can be time- 
consuming to 
undertake — design 


Surveys technologies and well suited to anticipating of questionnaires, 
ardiatien associated skills of future technological changes conducting fieldwork, 
primary interest. Surveys will tend | and future skill needs. cleaning data, 
data to provide information on producing findings. 
eoliections the extent of use of skills 
and technologies, extent to 
which skills are available, 
efforts taken to fulfil skill 
needs, etc. 
Where time series data are | Skills forecasting models If the model already 
available on skill needs tend to provide a projection | exists, analysis can 
(based on qualification and | of future demand, based be undertaken over 
occupation), and where on an extrapolation of a relatively short 
there is an underlying past trends and/or current space of time. But 
macroeconomic model policy. The assumption is setting up the initial 
Skills that can provide robust that the future is based on model and ensuring 
forecasting estimates of future a continuation of things as regular updates of 
employment demand by they are currently. Scenarios | the results can be 
sector, skills forecasts can | provide some basis for time-consuming and 
provide a robust means varying this to some extent, | resource-intensive. 
of providing quantitative to account for continual 
projections of future skill technological change. 
demand 
(circa 10 years ahead). 
Particularly useful where Can provide relatively Can be time- 
views about the future may | real-time information on consuming to 
not be well developed: technological change and develop initial search 
where there is uncertainty | skill needs. By identifying algorithms but, 
about either the types of those technologies that are | once established, 
technology that are likely at the point of take-off, there | can be undertaken 
to become dominant or is scope to gauge likely in a relatively fast 
Big data commonplace, and/or the _| future skill needs. There are | manner. It needs to 
analysis skills associated with those | uncertainties about how be borne in mind that 


technologies. Can also 
provide the detailed level 
of analysis that forecasting 
and surveys struggle to 
provide. 


representative data are of 

a given population and about 
how much ‘noise’ can be 
removed from any analysis 
or their inability to provide 
standardised information on 
skills complexity. 


coding/classifying 

of technology and 
skills data can be 
time-consuming. 
Maintenance and 
operational costs are 
also non-trivial. 
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Type of Capacity to mee 


Participatory approaches 


Technology 
foresight 


Where there is a large 
amount of information 

that needs synthesising to 
develop actions to ensure 
that skills needs associated 
with particular technologies 
can be met. 

Where there is limited data 
and information and where 
expert groups can address 
the lack of information. 


Can provide a view of the 
future and, importantly, 

an indication of how the 
future might be shaped for 
the benefit of society as 

a whole. Is dependent upon 
the availability of expert 
groups who can provide 
key input and a process in 
place to develop a degree of 
consensus about the future 
direction of change. 


Depends upon 

the scale of the 
exercise. Full-scale 
foresight involving 
a large number of 
participants is likely 
to prove time- 
consuming. But it 
is possible, and at 
times advisable, to 
conduct foresight 
with smaller groups 
over a relatively 
short-time span. 


Source: Cedefop. 
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Acronyms 


Al artificial intelligence 

Cedefop European Centre for the Development of Vocational Training 
CEPIS Council of European Professional Informatics Societies 

CSIRO Commonwealth Scientific and Industrial Research Organisation 
EU European Union 

FGB Fondazione Giacomo Brodolini 

ICT information and communications technology 

IT information technology 

LMSI labour market and skills intelligence 

MCA multiple criteria analysis 

O&A questions and answers 

R&D research and development 

SBS Straighten basic skills 

SME small and medium-sized enterprise 

STEEPV social, technological, economic, environmental, political and values 
STEM science, technology, engineering, mathematics 

STI science, technology and innovation 

SWOT strengths, weaknesses, opportunities and threats 

VET vocational education and training 
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Technology and skills foresight 


The world of work is being impacted by a fourth industrial 
revolution, transformed by artificial intelligence and other 
emerging technologies. With forecasts suggesting large 
shares of workers, displaced by automation, in need of 
upskilling/reskilling, the design of active skills policies is 
necessary. 

Conventional methods used to anticipate technological 
change and changing skill needs, such as skill surveys 
and forecasting, have limited scope to provide insights 
into emerging trends. With the increasing use of big data 
and Al methods, analysts have new real-time tools at their 
disposal. Skill foresight techniques are also increasingly 
used to gauge in-depth stakeholder information about 
future technologies and skill needs. 

A series of Cedefop guides aims to inform analysts and 
policy-makers about available skills anticipation methods 
used to navigate through the uncertainty of changing 
technologies and skill demands. This third practical guide 
focuses on technology and skill foresight methods. 
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